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Preface 



The first question we may face is: "Another Calculus book?" We feel that the an- 
swer is quite simple. As long as students find mathematics and particularly calcu- 
lus a scary subject, as long as the failure rate in mathematics is higher than in all 
other subjects, except maybe among students who take it as a major in College 
and as long as a large majority of the people mistakenly believe that only geniuses 
can learn and understand mathematics and particularly analysis, there will always 
be room for a new book of Calculus. We call it Calculus Light. 

This book is designed for a one semester course in "light" calculus, and meant 
to be used by undergraduate students without a deep mathematical background 
who do not major in mathematics or electrical engineering but are interested in ar- 
eas such as biology or management information systems. The book's level is suit- 
able for students who previously studied some elementary course in general 
mathematics. Knowledge of basic terminology in linear algebra, geometry and 
trigonometry is advantageous but not necessary. 

In writing this manuscript we faced two dilemmas. The first, what subjects 
should be included and to what extent. We felt that a modern basic book about 
calculus dealing with traditional material of single variable is our goal. The 
introduction of such topics and their application in solving real world problems 
demonstrate the necessity and applicability of calculus in practically most walks 
of life. 

Our second dilemma was how far we ought to pursue proofs and accuracy. We 
provided rigorous proofs whenever it was felt that the readers would benefit either 
by better understanding the specific subject, or by developing their own creativity. 
At numerous times, when we believed that the readers were ready, we left them to 
complete part or all of the proof. Certain proofs were beyond the scope of this 
book and were omitted. However, it was most important for us never to mix intui- 
tion and heuristic ideas with rigorous arguments. 

We start this book with a historical background. Every scientific achievement 
involves people and is therefore characterized by victories and disappointments, 
cooperation and intrigues, hope and heartbreak. All of these elements exist in the 
story behind calculus and when you add the time dimension - over 2400 years 
since it all started, you actually get a saga. We hope the reader enjoys reading the 
first chapter as much as we enjoyed the writing. 

In chapters 2-7 we present the topic of single variable calculus and these 
chapters should be studied in sequential order. The next two chapters provide ba- 
sic theory and applications of Fourier series and elementary numerical methods. 
They are expected to motivate the student who is interested in applications and 
practicality. 



VIII Preface 

The final chapter contains several special topics and combines beauty - the 
proof that e is irrational, and practicality - the theory of Lagrange multipliers in- 
troduced with a short introduction to multi- variable calculus. 

Each chapter is divided into sections and at the end of almost every section, va- 
riety of problems is given. The problems are usually arranged according to the or- 
der of the respective topics in the text. Each topic is followed by examples, simple 
and complex alike, solved in detail and graphs are presented whenever they are 
needed. In addition we provide answers to selected problems. 

It should be noted that the content of this book was successfully tested on many 
classes of students for over thirty years. We thank many of them for their con- 
structive suggestions, endless reviews and enormous support. 



Tampa, FL 2010 M. Friedman 

A. Kandel 
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1 Historical Background 



1.1 Prelude to Calculus or the Terror of the 'Infinite' 

Zeno, the 5-th century BC Greek philosopher, who is mainly remembered for his 
paradoxes, never gained the same prestige and admiration as did for example 
Socrates or Plato. But more than any other philosopher before or after him, Zeno 
introduced strong elements of uncertainty into mathematical thinking. He stumped 
mathematicians for over 2000 years, and his paradoxes provided both controversy 
and stimulation (besides entertainment of course) that inspired new research and 
ideas, including the development of calculus. This is quite fascinating considering 
the fact that he was a philosopher and logician, but not a mathematician. 

Born around 495 BC in Elea, a Greek colony in southern Italy, Zeno was a 
scholar of the Eleatic School, established by his teacher Parmenides. He quickly 
adopted Parmenides' monistic philosophy which stated that the universe consists 
of a single eternal entity (called 'being'), and that this entity is motionless. If our 
senses suggest otherwise, we must assume that they are illusive and ignore them. 
Only abstract logical thinking should count as a tool for obtaining conclusions. 

Determined to promote his teacher's concept of 'A// is One', Zeno invented a 
new form of debate - the dialectic, where one side supports a premise, while the 
other attempts to reduce the idea to nonsense. He also presented a series of 
paradoxes through which he tried to show that any allegation which does not 
support Parmenides' ideas, leads necessarily to contradiction and is therefore 
absurd. The keyword in the paradoxes was 'infinite'. According to Zeno, any 
assertion that legitimizes the concept of "infinity", openly or through the back 
door, is absurd and therefore false. Consequently, the opposite assertion is true. 

In the following paradox, Zeno argued that motion is impossible: If a body 
moves from A to 5 it must first reach the midpoint B^ of the distance AB . But 

prior to this it must reach the midpoint B2 of the distance AB^ ■ If the process is 
repeated indefinitely we obtain that the body must move through an infinite 
number of distances. This is an absurd and thus the body cannot move. 

The most famous of Zeno's paradoxes is that which presents a race between the 
legendary Greek hero Achilles and the tortoise. The much faster Achilles 
gracefully allows the tortoise a headstart of 10 meters. But his generosity is going 
to cost him the race no matter how slow the tortoise is. This astonishing 
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2 1 Historical Background 

conclusion is easily derived. Achilles must first pass the 10 meters gap. At that 
time the tortoise travels a shorter distance, say 1 meter. Now, before catching up 
with the tortoise, Achilles must travel a distance of 1 meter. The tortoise does not 
wait and advances another 0. 1 meter. If the process is repeated we realize that just 
to get to the tortoise, Achilles must travel an infinite number of distances, namely 

(10 + 1 + — + — - + ■■■) meters 
10 10^ 

which cannot be done in a finite time. 

Most philosophers and scientists were not comfortable with Zeno's paradoxes 
and did not accept his convictions. Aristotle discarded them as "fallacies" but did 
not possess any scientific tools to justify his conviction, as the basic concepts of 
calculus, the 'infinite series' and the 'limit', were yet unknown. Only the 
rigorously formulated notion of converging series as presented by the French 
mathematician Cauchy, and the theory of infinite sets developed by the German 
mathematician Cantor in the 19-th century, finally explained Zeno's paradoxes to 
some satisfaction. 



1.2 Calculus - Where Do We Start? 

Some people claim that it was Sir Isaac Newton, the 17-th century English 
physicist and mathematician who discovered calculus. Others give the credit to the 
German philosopher and mathematician Baron Gottfried Wilhelm von Leibniz, 
but most believe that both scientists invented calculus independently. Later we 
will discuss this particular dispute in length. One fact though is above dispute. All 
agree that discovery of calculus took place between the years 1665 and 1675 and 
that the first related publication was Leibniz's article Nova Methodus pro Maximis 
et Minimis ("New Method for the Greatest and the Least") which appeared in Acta 
Eruditorum Lipsienium (Proceedings of the Scholars of Leipzig) 1684. 

Yet, to think that this unique mathematical field started from scratch in the 
second half of the 17-th century is incorrect. Various results closely related to 
some of the fundamental problems and ideas of calculus, particularly the integral 
calculus, has been known for thousands of years. The ancient Egyptians for 
example who were practically involved in almost every important development in 
culture and science, knew how to calculate the volume of a pyramid and how to 
approximate the area of a circle. 

Then came Eudoxus. Born about 400 BC in Cnidus, southwest Asia Minor, this 
brilliant Greek studied mathematics and medicine (not an unusual combination for 
those days), in a school that competed with Hipocrates' famous medical school in 
the island of Cos. A local rich physician, impressed by the talented youngster paid 
Eudoxus's way to Plato's Academy in Athens. Eudoxus then spent some time in 
Egypt where he studied astronomy. Later he was a traveling teacher at northwest 
of Asia Minor and finally returned to Athens where he became an accomplished 
legislator until his death (about 350 BC). 
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The best mathematician of his day, Eudoxus is remembered for two major 
contributions in mathematics which are extensively discussed in the thirteen books 
collection Elements by Euclid (second half of 4-th century BC). The first is the 
Theory of Proportions which paved the way to the introducing of Irrational 
Numbers and is discussed in Book V. The second is the Method of Exhaustion, 
presented in Book XII, which was developed to calculate areas and volumes 
bounded by curves or surfaces. The general idea of this method was to 
approximate an unknown area or a volume using elementary bodies like rectangles 
or boxes, whose areas or volumes were already known. At each stage, at least half 
(or other constant fraction) of the remaining area must be replaced by new 
elementary bodies. This guarantees that the unknown area may be approximated to 
any desired degree. It was the first time that the concept of infinite simally small 
quantity was included, over 2000 years before Newton and Leibniz. It is therefore 
quite legitimate to consider Eudoxus as the discoverer of the conceptual 
philosophy of integral calculus. 

The true successor of Eudoxus and his method of exhaustion was Archimedes 
of Syracuse (287-212 BC). A first class inventor, the founder of theoretical 
mechanics and one of the greatest mathematicians of all times, Archimedes 
studied in Alexandria, capital of Egypt and center of the scientific world at the 
time. After completing his studies he returned to Syracuse where he remained for 
the rest of his life. The Greek biographer Plutarch (about 40-120 AD) claims that 
Archimedes despised everything of practical nature and rather preferred pure 
mathematical research, whether it was solving a complex problem in geometry or 
calculating a new improved approximation to the circumference of a circle. Yet, 
when forced by the circumstances, his beloved city under Roman siege, he put his 
engineering ingenuity to work and devised war machines for Hieron II, King of 
Syracuse, which inflicted heavy casualties on the Roman Navy. When the city 
finally surrendered to the Romans, Archimedes, realizing that everything was lost, 
went back to his research. Unfortunately there was not much time left for him. 
While engaged in drawing his figures on the sand he was stabbed to death by a 
Roman soldier. 

As stated above, Archimedes adopted and improved the method of exhaustion 
originated by Eudoxus and followed by Euclid. One of his applications was to 
approximate the area of a unit circle (equals to 2n) by using inscribed and 
circumscribed equilateral polygons to bound the circle from below and above. It is 
an early example of integration, which led to approximate values of ;r . A sample 
of his diagram is given in Fig. 1.1.1. 

By increasing the number of the vertices of each polygon, the interval where 
the exact value of the area is located decreases, and the approximation improved. 

At some point Archimedes believed that k equals VlO . However, after using the 
method of exhaustion, he found the assumption false and by taking 96 vertices 
showed 

3l^<.<3i 
71 7 
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Fig. 1.1.1. Approximating the circle by equilateral polygons. 

Around the year 225 BC, already over sixty, Archimedes published his book 
Quadrature of the Parabola where he demonstrated that any portion of a parabola 
confined by a chord AB (Fig. 1.1.2) is 4/3 of the triangle with base AB and 
vertex C (obtained by drawing the tangent parallel to AB ; the segment 
connecting the midpoint M of AB with C is parallel to the parabola axis of 
symmetry). Archimedes constructed a series of increasing areas, each composed 
of triangles, which approximated the parabola to any desired degree. If A denotes 

the triangle's area, then at the M-th step, 2" new triangles of total area A I A" were 
added to the (« - 1) -th approximation. 




Fig. 1.1.2. A(A5C) + A(CBCi) + A(ACC2) = A + A/8 + A/8 = A + A/4 



1.3 The Countdown 5 

The series of approximations is 

A,A + — ,A + — + —-,... 
4 4 4^ 

and it converges to (4/ 3) A, the area of the parabola. This was the first known 

example in history that summation of an infinite series took place and was 
documented. 



1.3 The Countdown 

No real breakthrough in the development of calculus occurred within the next 
1700 years between Archimedes' death and early 16-th century. However, during 
the late middle ages, mathematicians studied Archimedes' treatises translated from 
Arabic and the concept of 'the infinite' which confused the Greeks, started to gain 
some respect and recognition. From late 16-th century on, an increasing number of 
scientists, often driven by problems in mechanics and astronomy, made significant 
contributions that led to the creation of calculus several years later. 

In his work on planetary motion, J. Kepler (1571-1630), the German 
astronomer, had to find areas of sectors of an ellipse. He considered an area as a 
sum of lines - a crude way of integration. Not bothered with rigor he was 
nevertheless lucky and got the correct answer, after making two canceling errors 
in his work. 

The Italian mathematician B. Cavalieri (1598-1647) stimulated by Euclid's 
work and a disciple of the famous astronomer Galileo, published his Geometry by 
Indivisibles where he expanded Kepler's work on calculating volumes. He used 

'indivisible magnitudes' to calculate the areas under the curves y = x" ,1<«<9. 
His approach though, observing an area like Kepler, as a collection of lines, was 
not rigorous and drew criticism. He also formulated a theorem (now called 
Cavalieri 's theorem) which determines the volume created by a rotating figure 
confined between two parallel planes. 

The French mathematician G. P. Roberval (1602-1675) considered the same 
problems as Cavalieri, but conducted a much more rigorous study. He treated the 
area between a curve and a straight line as an infinite number of 'infinitely 

narrow' rectangles and then applied his scheme first to approximate the area of x" 
from to 1, then to 'obtain' this area as the value l/(n + l), to which the 

approximates were approaching. 

Two other French mathematicians are credited for major contributions 
during these pre-calculus years. R. Descartes (1596-1650), a philosopher and 
mathematician, is unanimously considered the inventor of coordinate geometry 
(today - analytic geometry). His goal was to unify algebra and geometry, mainly 
to apply algebraic methods for solving problems in geometry. He was the first to 
classify curves by the equations which produced them. An independent discoverer 
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of analytic geometry was P. Fermat (1601-1665) mainly remembered for 
Fermat's last theorem and other contributions in modern theory of numbers. 
Fermat calculated the greatest and the least values of algebraic expressions using 
methods similar to those of modern calculus and finding when the tangents to the 
associated curves were parallel to the x -axis. This is why some consider Fermat 
the real inventor of calculus. 

An English mathematician, J. Wallis (1616-1703), published Arithmetica 
Infinitorum (Arithmetic of Infinitesimals), a book in which he showed how to use 
features that hold for finite processes to extrapolate formulas for infinite 
processes. He also introduced the symbol 0° for 'infinity'. I. Barrow (1630-1677) 
who was also Newton's teacher, obtained a method for determining tangents, that 
was very close to the methods used in calculus. In his book Lectiones Geometricae 
(Geometrical Lectures) published in 1670, Barrow provided a geometrical 
presentation of the inverse relationship between finding tangents and areas. 

By the late 17-th century there were strong indications that calculus was just 
beyond the horizon. 

1.4 The Birth of Calculus 

There is a general consensus that Newton and Leibniz contributed more than 
anyone else to the development of calculus, and are its inventors. It should be 
noted though, that 150 years after Newton and Leibniz, the foundations of calculus 
were still shaky. Only in 1821, the French mathematician A.L. Cauchy 
(1789-1857), finally removed the remaining logical obstacles. His 'theory of 
limits' did not depend on geometric intuition, but gave a rigorous, logical 
presentation of basic concepts as continuity, derivative and integral. Fortunately 
however, the lack of rigorous formalism in its early stages, did not prevent 
calculus from quickly becoming the most important and effective mathematical 
tool in the development of modern science. 

For both Newton and Leibniz, calculus had not been the sole or even the main 
subject of interest. Nevertheless, the question who deserved credit for the 
significant discovery, later became a major source of fury and agony for the two 
scientists, casting a giant shadow over their remaining days. 

Born in 1642 at Woolsthorpe in Lincolnshire, England, Isaac Newton did not 
enjoy a happy childhood. As if his father's death, three months before he was 
born, was not enough, his mother soon married an elderly minister and left the 
2-year old boy with his grandmother. Only when widowed for the second time did 
she take him back at the age of eleven. There is no doubt that these traumatic 
events, could account at least partially, for Newton's general ill temper, his 
occasional nervous breakdowns and his negative attitude towards women. His 
scientific genius however, was not affected. 

Soon after Newton's reunion with his mother, he was appointed the manager of 
her estate. Fortunately, the youngster's uncle. Reverend William Ayscough 
interfered. A Cambridge scholar and a man of authority, Newton's uncle 
convinced his sister that her son's future should not end at Woolsthorpe. After 
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studying in the grammar school at Grantham Newton arrived in Cambridge and 
graduated with a BA in 1665. Due to an outburst of the plague, he returned home 
for almost two years. This period was the most fruitful and creative in Newton's 
or in any other scientist's life throughout human history. 

Within 20 months, Newton laid the foundations to calculus, optics and 
gravitation. He disclosed his results to close friends but unfortunately did not 
publish them. Only in 1687, some of Newton's discoveries, including his theory of 
gravitation, appeared in the first edition of his Philosophiae Naturalis Principia 
Mathematica (Mathematical Principles of the Natural Philosophy). It took him 
additional 17 years to publish the details of his 'Fluxtional Method', which he 
used to lay the foundations of calculus. Newton's approach to calculus was 
dynamic, observing variables as flowing quantities generated by continuous 
motion of points. He referred to a variable quantity denoted for example by X as 
'fluent' and to a variable's rate of change, which he denoted by i: , as 'fluxion'. 
The two fundamental problems of calculus as regarded by Newton were: Given a 
relation between two fluents, find the relation between their respective fluxions 
and vice versa. 

Newton's unique contributions to modern science were just one aspect in his 
life. He was always interested in theology as well and involved also, particularly 
in later years, with alchemy and mysticism. A professor in Cambridge from 1669, 
Newton retired in 1696 to become warden and then master of the Royal Mint in 
London. In 1703 he was elected president of the Royal Society. Knighted by 
Queen Anne in 1705, it was the first time a scientist was ever awarded this honor. 

Unable to confront criticism, accept a colleague's success or share credit for 
discoveries with anyone, Newton's relations with other scientists were often 
disrupted permanently. Nothing though compares with the violent crusade he 
carried against Leibniz, over the glory of inventing calculus. This dispute 
dominated the last 25 years of his life, until his death in 1727. It is an appalling 
story which glorifies neither the Newton, nor the Royal Society, for its 
unconditional surrender to its president's unbalanced behavior. However, before 
reviewing the details of this dispute, let us present the other contestant: A scientist 
of Newton's caliber, who also found himself at the center of the most famous, 
bitter and quite scandalous scientific priority dispute of modern time. 

If anyone should be regarded as 'universal genius', it is Gottfried Wilhelm 
Leibniz, the late 17-th century philosopher and mathematician. He influenced every 
possible discipline of natural and social sciences. Born in Leipzig, Germany in 1646, 
Leibniz lost his father, a professor of moral philosophy, at the age of 6 (compare with 
Newton). At the age of 15 he entered the University of Leipzig as a law student, and 
received the degree of doctor of law at 20. He turned down an offer of a professor's 
chair at the University of Altdorf. Instead, Leibniz pursued a political career and 
became a skillful diplomat at the service of the Archbishop of Mainz. 

During four years in Paris (1672-76) Leibniz was intellectually stimulated by 
constant interaction with leading philosophers and scientists. He particularly 
benefited from meeting the Dutch scientist C. Huygens, who enriched his 
mathematical knowledge and thinking. In Paris he also devised a calculating 
machine. While an adding and subtracting machine was already at hand, Leibniz 
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designed an elegant device which mechanized multiplication - another aspect of 
his genius. This device was still found in calculating machines in the 1940's! 

In 1676 Leibniz returned to Germany and settled in Hanover for the rest of his 
life as a Counselor of the Duke of Brunswick - Luneberg. His trivial duties, such 
as composing the family tree of the Brunswicks, left Leibniz plenty of time for 
more serious research in philosophy and mathematics. His major work in 
philosophy, published in early 18-th century, describes the universe as a collection 
of 'monads'. Each monad is a unique, indestructible dynamic substance, 
distinguished from other monads by its degree of consciousness. 

Leibniz's most important contribution to mathematics and to science, in 
general, was laying the foundations to calculus. The fruitful discussions with 
Huygens gave him the insight that the operations of summing a sequence and 
taking its differences sequence, are in a sense inverse to each other. He also 
realized that quadrature could be observed as summation of equidistant ordinates 
and that the difference of two consecutive ordinates approximates the local slope 
of the tangent. Finally, if the distance between successive ordinates becomes 
'infinitely small', this approximation becomes exact. Leibniz did not really 
explain the concept 'infinitely small', but introduced the notations which 
were adopted forever by the mathematical world: dx,dy for the 'differentials', 

dy/dx for the derivative and a long S written as for the integral. Leibniz 

published his discovery first in 1684 followed by a second article about integral 
calculus in 1686. 

Unhappy with his post, the quarrel with Newton over the invention of calculus, 
only added to Leibniz's bitterness and frustration. Mistreated by the princes whom 
he served and condemned to remain in the provincial Hanover, he died lonely and 
forgotten in 1716. 



1.5 The Priority Dispute 

The scientific world never witnessed such a long, vicious and paranoid battle as 
that of Newton and Leibniz. This may seem strange, since scientists supposedly 
portray an image of sincerity, objectivity, moderation, and even modesty. But 
scientists, like others, struggle for kudos often at the expense of the truth. 

Three hundred years after the stormy priority dispute over calculus, it is 
unanimously agreed that while Newton was the first to lay the foundations of 
calculus, Leibniz, who discovered it ten years later using a different approach, was 
the first to publish his results. For years mutual respect existed between the two 
scientists. But misinterpretation of certain documents, half true hearsay and 
ignorance about each other's work, led to open accusations and unrestrained 
attacks. While Leibniz probably never denied Newton's unique contribution, 
Newton's arrogance and obsession for continued praise, prevented him from 
crediting Leibniz. Unable to accept the fact that the first scientific publication 
announcing the birth of calculus was not his, he sent the Royal Society in 1699, a 
communication accusing Leibniz of plagiarism. 
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In 1712, the society, totally controlled by Newton, appointed a special 
committee to look into this issue. Staffed with five Newtonians, the committee not 
only supported Newton's priority claim, but officially accepted the plagiarism 
charge. The humiliated Leibniz tried to fight back. For some time he trusted the 
Berlin Academy to support him, but fighting the Royal Society was unpopular and 
the academy had neither troops nor desire to do so. Not even Leibniz's death 
halted Newton's obsession. He once arrogantly told an admirer that "he broke 
Leibniz's heart with his reply to him". Ironically, it was Leibniz who 
complemented his opponent: "Taking mathematics from the beginning of the 
world to the time of Newton, what he has done is much better than half . . . 

We would like to close the historical background with the following, somewhat 
related, 'optimistic' episode. Sometime in early 19-th century, in a little pub in 
Gottingen, Germany, students of Goethe and Schiller, the most prominent German 
poets, were arguing which of the two was greater. As beer was spilling, the debate 
was warming up, almost getting violent. Finally, moments before swords were 
drawn, one peacemaker suggested to bring the issue before Goethe himself, 
known for his objectivity and trusted by everyone to pass an unbiased judgment. 
Goethe listened quietly and then, so they say, turned towards the students with a 
smile, mixed with sadness and irony and said: "Gentlemen, 1 am amazed. Think 
how fortunate you all are, to have both Goethe and Schiller in your generation. 
And here, instead of being thrilled and thankful, all that interests you is who is 
number one..." 



2 The Number System 



The basics, which are essential for the study of calculus, are Numbers and 
Arithmetic Operations. However, we feel that prior to introducing the number 
system, we should familiarize the reader with some helpful elementary concepts 
from set theory. 

2.1 Basic Concepts about Sets 

A set is a collection of distinct objects, which can be observed as one entity. Each 
object is called element or member of the set. Each element of a set is said to 
belong to the set, while the set is composed of its elements. Examples of sets are 
found everywhere: a family, a forest, a class. A family is composed of persons; a 
forest is composed of trees; a class is composed of students. 

Sets are usually denoted by capital letters: A, B, C,...; elements are usually 

denoted by lower-case letters: a,b,c,... . The fact that an element X belongs to a 

set S , is expressed by the notation 

xe S 

If the element X is not a member of 5 , we write xi S . The relation between a 
set and an element is very crispy: the element either belongs to the set or does not 
belong to it. Another convenient notation is displaying the elements of a set in 
braces. For example a set whose elements are the numbers 0,-5,17 will be 

written as {0,-5,17}. 

A set whose elements are all the positive integers from 1 to 1000, will be 
written as jl, 2, 3, . . . ,1000) , and a set consisting of all the positive integers will be 

written as {l, 2, 3, ...} or as {l, 2, 3, ...,n,...}. Note that {0,-5,17} may be also 
written as {-5,17,0} , i.e. the order of the elements is irrelevant. What counts is 

membership. This will become clearer as we define the basic relations between 

sets. 

Definition 2.1.1. Two given sets A,B are said to be equal, i.e. A = B , if each 

element of A belongs also to B and if each element of B belongs also to A . This 
can be also written in short notation as xe A^' xe B and x g fi => jc 6 A , or 
even shorter asjcGA<=^xG5.If the two sets are not equal we write A^ B . 
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For example, {0,-5,17} = {-5,17,0j . 

Definition 2.1.2. A set A will be called a subset of a set B , if every element in 
A is also in B , i.e. if XGA=>xeB.In this case we write A c Z? . If there is an 
XG B such that xi A,we write Acz B . 

For example, let A = {Mom, Dad, Jack, Martha,Tom} and B = {Mom,Tom} then 
B is a subset of A , or 5 c A . Note that here B c A as well. 

Theorem 2.1.1. Let A,B be arbitrary sets. Then 

A = B ifandonlyif A ^ B and B ^ A (2.1.1) 

The proof is trivial and follows from the previous definitions. 

Definition 2.1.3. For arbitrary sets A,B we define the union of A and B as the 

set C , composed of all the elements which belong to either A or Z? . We write 
C=AuB. 

Definition 2.1.4. For arbitrary sets A, B we define the intersection of A and B 
as the set C , composed of all the elements which belong to both A and B . We 
write C = AnB . 

Definition 2.1.5. For arbitrary sets A, B we define the difference of A and B as 
the set C , composed of all the elements which belong to A but not to B . We 
write C = A-B. 

Example 2.1.1. A prime number is a positive integer which has no factor except 
itself and 1. Let A denote the set of all the primes between 3 and 18 and let B 
denote all the odd numbers between 12 and 20, i.e. A = {3,5,7,11,13,17} and 
B = {13,15,17,19}. 
Then 

AuB = (3,5,7,1 1,13,15,17,19}, AnB = {13,17} 
A-B = {3,5,7,11}, B- A = {15,19} 

If a set has no elements, it is called the empty set and is denoted by . The sets 
A and B are called disjoint if A n B = . The notations of union, intersection 
and difference, are illustrated in Fig. 2.1.1. 
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(a) A u B 



(b) Ar^B 
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(c) A-B (d) disjoint sets 

Fig. 2.1.1. Union, intersection and difference of sets. 



A popular way for defining a particular set is by writing 

S ={x\ X satisfies P] 

It means that an element x belongs to S if and only if x satisfies some property 
P . For example, let S ={x\ x is a positive integer less than 4}. Then S contains 
the numbers 1, 2 and 3 and will be written as S ={1,2,3} . Quite often several 
properties are needed to define a set. For example, the set of all positive numbers 
whose square is less than 7 may be presented as {x I x > , x < 7} . 

The operations 'union' and 'intersection' are commutative, i.e. arbitrary 
sets A, B satisfy Akj B = Byj A and A(^B = B r\A . They are also associative, 

i.e. for arbitrary sets A,B,C we have (AuZ?) uC = Au(SuC) and 
{Ar^B)r^C = Ar^(B r^C) . These results are straightforward. Consider for 
example the assertion {Ar^B)r^C = Ar\(Br\C) . We have to show that each 
side is a subset of the other. Indeed, if x& (A(^B)r^C then x g A n B and 
xe C . Thus xe A and xe B as well. Consequently xe B niC and since xe A 
we obtain x e A (^ (B r\ C) . Therefore, the left-hand side of the relation is a 

subset of its right-hand side. The opposite is shown similarly. 

We will now extend the notations of union and intersection for any number of 

sets. 
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Definition 2.1.6. Let 3 denote a 'class' of sets and let A denote an arbitrary set 
in 3 . The union of all the sets in 3 is the set which contains every x that 

belongs to at least one set A . It is denoted by I J A . The intersection of the sets 

in 3 is the set which contains every x that belongs to all A in 3 . It is denoted 
\A. 

Ae3 



byp^ 



PROBLEMS 

1. Which of the following collections are sets? Which sets are equal? 
(a) {-4,7,3,-3} (b) {-3,7,-4,-3} (c) {7,3,-4,-3} 

2. Let A = {x I < ;«: < 4} , 5 = {x I 3 < X < 5} . Obtain Au B , AnB , A- B . 

3. Show (A-B)n(B-A) = for arbitrary sets A,B . 

4. (a) Show that is a subset of any given set. (b) Find the set S whose 
elements are all the subsets of {1,2,3} . How many elements did you get? 

5. For arbitrary sets A,B,C prove the distributive laws: 

An(B^C) = (AnB)Kj(AnC) 
Au(5nC) = (AuB)n(AuC) 

6. For arbitrary sets A, B, C show A - (B n C) = ( A - B) u ( A - C) . 

7. For arbitrary sets A, B, C show A - (B u C) = ( A - B) - C . 

8. For arbitrary sets A,B : A u ( A n B) = A n (A u B) = A . 

9. Let 3 denote a class of sets and let B denote an arbitrary set. Then 

B-\jA=f](B-A) , B-P|A=(J(B-A) 

Ae3 Ae5 Ae5 Ae5 

2.2 The Natural Numbers 

The 19-th century German mathematician Leopold Kronecker once said: "God 
created the natural numbers. All the rest is man-made." Indeed, it is impossible to 
imagine a world without the natural numbers. A world where simple questions 
like "How many children have you got?" or "When do you expect the next full 
moon" would be so difficult to answer. The rigorous formalism for introducing the 
set A'^ of natural numbers is based on the following four axioms i.e. statements 
which intuitively are true but we are unable to prove: 
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1 . If the successors of n and m are equal, then n = m . 

2. Every natural number ne N has another associated unique natural 
number n'e N , called the successor of n . 

3. There is a number called 1, which is not a successor of any natural 
number. 

4. If a subset M of A^ is such that the two requests: (a) le M ; and (b) If 
riE M then n'e M are satisfied, then M = N . 

These are the Peano axioms and their acceptance is the foundation for the natural 
number system. We denote the successor of 1 by '2', the successor of 2 by '3' and 
thus obtain the familiar natural numbers. 

Axiom 4 is called the Principle of Mathematical Induction. It provides a 
process by which we can determine, whether a statement P{n) associated with the 
natural number n , is valid for all the natural numbers. The process consists of: 

(a) Showing that P(l) is true. 

(b) Showing that if P{n) is true, than P{n') is true as well. 

If both (a) and (b) are satisfied, then by virtue of Axiom 4, the set of natural 
numbers for which _P holds, is identical to A^. If instead of (a) we show 
thatP(Mo)is true for some natural number Mq and that (b) holds as well, then P 
must be true at least for all the natural numbers greater or equal to Mq . 

Every result associated with natural numbers, follow from the Peano axioms. 
For example. Axiom 3 guarantees that A^ is nonempty: at least 1 belongs to A^ . 
Another example is the statement that the numbers 1, 2, 3 are all different. This is 
shown as follows: 

(1) 2^1: Otherwise 2 = l^r=l^>l is a successor of a natural number, 
contradicting Axiom 3. 

(2) 3 ;^ 1 : Otherwise 3 = 1^2'=1^> again contradicting Axiom 3. 

(3) 3^2: Otherwise 3 = 2^> 2'= r^> 2 = Iby Axiom 2. However, this is 
impossible as we have seen in part (1). 

By using induction (Axiom 4) we can show that all the natural numbers are 
different. 

In order to 'calculate' with natural numbers, we introduce 'arithmetic 
operations' between them. We define addition (+) and multiplication (•) and 
expect them to possess the following elementary familiar properties: 

n + l = n' , n + m'=(n + my (2.2.1) 

nl = n , n-m'= n-m + n (2.2.2) 

Do such operations exist? This is answered next. 
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Theorem 2.2.1. There exist unique operations + , • between the natural numbers 
such that Eqs. (2.2.1-2) are satisfied for all n,me N . 

We usually use the shorter notation nm for multiplication. Addition and 
multiplication satisfy the basic rules of arithmetic: 

m + n = n + m , mn = nm (commutative laws) 

{m + n) + l = m + {n + l) , {mn)l = m(nl) (associative laws) 
(m + n)l = ml + nl , m(n + l) = mn + ml (distributive laws) 

all of which can be verified using induction and Eqs. (2.2.1-2). For example, let us 
obtain the associative law for addition. 

Theorem 2.2.2. For arbitrary n,m,le N we have 

{m + n) + l = m + (n + i) (2.2.3) 

Proof. 

For arbitrary prefixed n,mE N we will apply induction to show the validity of 
Eq. (2.2.3) for all le N . 

(a) 1 = 1: By virtue of Eq. (2.2.1) (m + «) + l = (m + «)'= m + M'= m + (M + l) 

(b) The assumption {m + n) + l = m + {n + 1.) combined with Eq. (2.2.1) yield 

{m + n) + r= [{m + n) + /]'= [m + {n + /)]'= m + {n + /)'= m + {n + /') 

which concludes the proof. 

The following result, the trichotomy law, defines order among the natural 
numbers. 

Theorem 2.2.3. For arbitrary n,mE N one and only one of the following 
relations is valid: 

(1) m = n 

(2) m = M + / , for some le N . 

(3) « = m + / , for some le N . 

If case (1) holds, the numbers are equal ; if case (2) holds, m is greater than n 
and we write m> n; if case (3) holds, m is less than n and we write m<n . 
Theorem 2.2.3 combined with the basic arithmetic rules, leads to many results 
which we usually take for granted. For example 
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m>n=>m + k>n + k for all k e N 
Indeed, m> n^> m = n + l for some le N . Therefore 

m + k = (n + l) + k = n + (l + k) = n + (k + l) = (n + k) + l 
i.e. (by definition) m + k > n + k . 

PROBLEMS 

1 . Obtain the commutative law for addition, m + n = n + m, by prefixing m and 
applying induction on n . 

2. Obtain the associative law for multiplication, {mn)l = m(nl) , by prefixing 
m, n and applying induction on / . 

3. Obtain the commutative law for multiplication, mn = nm . Hint: to show 
m-\ = \-m apply a second induction on m . 

4. Show : m>n,k>l=>m + k>n + l, mk > nl . 

5. Show: m>n,k>l=>m + k>n + l, mk > nl . 

6. Show: m>n,k>l^>m + k>n + l,mk>nl. 

7. Show: (a) m > n=> mk > nk (b) mk > nk => m> n . 

8. Define n =n-n and apply the basic arithmetic rules to show: 

(a) (m + 1)^ = n^ + 2m + 1 (b) {n + m)^ = n^ + 2nm + m^ . 

9. Show n^ >n + 3 for all n>2. 
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A simple equation such as 3 + x = 7 , where x is some unknown natural number, 
is solved uniquely by x = 4 . However, other equations, just as simple, for 
example 5 + y = 2 ,or 6z = 13 , have no solutions, i.e. there are no y or z for 
which these equations hold. Nevertheless, these particular relations may simulate 
real 'situations'. Consequently, unless determined to significantly limit our 
capacity to solve real-world problems, the concept of number must be extended 
beyond the set of the natural numbers. In this section we discuss two extensions 
and introduce the integers and the rational numbers. However, prior to this, we 
ought to introduce the concept of 'extension' which is often used throughout this 
chapter. Let A and B denote sets of 'numbers' and let Pj^ and Pg be 'arithmetic 
operations', which are defined over A and B respectively. The set B will be 
called an extension of A , if either 
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(a) (1) A c B , (2) xPgy = xP^y for all x,y^A ; or 

(b) There exists a subset A<z B such that 

(1) There is a one-to-one correspondence between A and A' . 

(2) If X, yeA correspond to x',y'&A' then x' P^y'^ A and x/'^}' 
corresponds to x' P^y' . 

In constructing the whole number system we use both types of extensions. 

2.3.1 Integers 

We add new objects to N : -n (called minus n) for each n& N and (called 
zero). 

In order to calculate with them, we extend the arithmetic operations as follows. 

Definition 2.3.1. For arbitrary m,nE N 

m + (—m) = 

m + (-n) = -l ifn = m + l , IsN (2.3.1) 

m + (—n) = I if m = n + l , I e N 



and 



(-n) + m = m + (-n) (2.3.2) 

(-m) + (-«) = —(m + n) 



Definition 2.3.2. For arbitrary m,nE N 

m-(-n) = (-n) ■m = —(m- n) (2.3.3) 

(— m) • (-«) = m-n 

Definition 2.3.3. For arbitrary ne N 

+ n = n + = n 

+ (-n) = {-n) + = -n (2.3.4) 

-(-«) = n 
and 

0-m = m-0 = (-m)-0 = 0-(-«) = (2.3.5) 

Definition 2.3.4. + = 00 = -0 = 0. 
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The elements -n,nE N are the negative integers. We refer to the union of the 

natural numbers, also called the positive integers, the negative integers and 0, as 
the integers. It can be shown that all the integers are different. The set of all the 
integers is denoted by / . The next result confirms the validity of the basic arithmetic 
rules for all the integers and supports the formalism used for extending N . 

Theorem 2.3.1. The commutative, associative and distributive laws for addition 
and multiplication hold over / . 

The equation m + x = n which did not always possess a solution in N , and 
provided the motivation to legitimate the negative integers and 0, has a unique 
solution x = n + (-m) for arbitrary m, n e / . We usually write n + (-m) = n-m 
and call this operation subtraction. For example 7 + (-1 1) = 7 - 1 1 = -4 . Finally, 

we leave it for the reader to show, that the procedure for obtaining the integers 
from the natural numbers, is an extension of type (a). 

2.3.2 Rational Numbers 

For every ordered pair of integers (m,n) such that « ;^ , we define an object 

m 
m/n or — , which we call a fraction. It is positive if m>0,n>0 or if 
n 

m <0, n <0 and we write m/n > . It is negative and we write m/n < if 
m > 0,M < or if m < 0,« > . 

It is zero if m = and we write m/n = . Equality and the arithmetic 
operations between fractions are defined as follows. 

Definition 2.3.5. For arbitrary fractions a/b and c/d 

(2.3.6) 



(2.3.7) 
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(2.3.8) 



This definition, which goes back to our early schooldays, is not only intuitively 
acceptable, but also guarantees the validity of the commutative, associative and 
distributive laws for fractions. In addition, we can show that for arbitrary fractions 
r, S one and only one of the following relations holds: 
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1 . r = s 

2. r = s + t , for some fraction f > . 

3. s = r + t , for some fraction f > . 

In case (2) we say that r is greater than s ; in case 3 we say that r is less than s . 

Note that equahty between fractions does not necessarily mean that they are 

identical. For example, 2/3 = 4/6 = 300/450 . Consequently, an equation such as 

2/5 + X = 1/2 has an infinite number of solutions, namely, 1/10, 2/20, (-4)/(-40) 

etc. 

Disturbing? Not really! An elegant way to remove this confusion is to create a 
set, called a rational number, whose elements are all the different equal fractions. 
For example the fraction (-2)/(-6) belongs to the rational number 

{mju I mjn = 1/3} . 

For all practical purposes, such as arithmetic operations, it is irrelevant which 
fraction of the set is used. We can therefore identify a fraction that solves a given 
equation, with the rational number to which this fraction belongs. Consequently, 
there are unique rational numbers which solve 

- + x = - (2.3.9) 

b d 

^.x = - a^O (2.3.10) 

b d 

The equal fractions {-a)/b , a/{-b) are denoted by -(a/b) . Using this notation 
we can write the solution to Eq. (2.3.9) as c/d + [-(a/b)] and define the operation 
'subtraction' as 

^-^ = ^J-^] (2.3.11) 

d b d { bj 

The solution to Eq. (2.3.10) is (bc)/{ad) (check by substituting). This motivates 
to define the operation 'division' as 



be 

a^O (2.3.12) 



a] ad 
b 



The left-hand side of Eq. (2.3.12) is the quotient of the two fractions. 
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n 
Note that there is one-to-one correspondence, n<-^— , between all the integers 

n 
n ,ne I and the fractions — . This correspondence is preserved under addition 



and multiplication, i.e. 



m n m n 

m + n<r^ 1 — , m-n<r^ 

11 11 



Indeed 



m + n m-\ + n-\ m n 

m + n <r^ = = 1 — 

1 Mil 



m- n m- n m n 
m- n <r^ 



11 1 1 

n 
Consequently, each rational number — can be replaced by the corresponding 

integer n in any computational process. 

The set of all the rational numbers is denoted by R At was derived from the set 
of integers, using an extension of type (b). We will also use /?"*" , R to denote the 
sets of all the positive and negative rational numbers, respectively. 



PROBLEMS 



1. Use induction to show I + 2 + 3 + ... + m = [«(« + !)]/ 2 for all ne N . 

2. Use induction to show P +2^ +3^ +... + «^ =[n^(n + lf]/4- for all ne N . 

3. Use induction to define 2" and show 2" > n for all ne N . 

4. Define 1!= 1 ; n\= « • (« - 1)! , « > 1 and show n\> n , « > 4 . 

5. Show the validity of the commutative laws for fractions. 

6. Show the validity of the associative laws for fractions. 

7. Let r, s denote fractions such that r < s . Find a fraction t :r <t < s . 

8. Let re R , R = {r\ r- fraction] . Define r" ,ne N and show 



l + r + r^ +... + r" = , ne N , r^[ 

r-\ 
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9. Show (1 + r)" >l + nr , ne N (Bernoulli's inequality) for arbitrary fraction 

r >-l. 



2.4 Real Numbers 

By introducing the rational numbers we can now apply the four arithmetic 
operations over R with the single restriction of dividing by 0. In addition we 
define integer powers of a fraction as 

r^V.r =i;r =r-r ,nEJy 
r ^0:r'" = — ,neN 

n 

r 

and 0" = , « e A^ . At this stage it may seem that additional extension of the 
number system is unnecessary, particularly since computers take and provide only 
rational numbers. Yet, a strong indication against this attitude is given simply by 
showing that some legitimate numbers are still missing from R ! For example, the 
area of the unit circle, denoted by ;r , is not a rational number, i.e. cannot be 
expressed as a fraction. The proof of this allegation is very complicated and is 
beyond the scope of this book. Instead, let us verify an equally important though 
much simpler statement. 

Theorem 2.4.1. There is no fraction x such that x =2 . 
Proof. 

2 / 2 

If such a fraction exists, then m /n =2 for some integers m,n . We may assume 

that these integers have no common factor. Since m =2n we conclude that m 
is even. Therefore m itself is even and we may write m = 2l for some integer / 

and obtain 2/ = n . A similar argument shows that n is even too, which 
contradicts the assumption that m,n have no common factor. Thus, the integer 2 

is not a square of any fraction. 

An immediate consequence of Theorem 2.4.1 is that we can either extend the 

number system to include objects such as -J 2 or ;r , or maybe deny their 
existence. The latter choice, though simpler, leads to an illogical situation. Indeed, 
consider a right angled isosceles triangle whose equal sides are of length 1 (Fig. 

2.4.1). By applying Pythagoras theorem we get AC^ = AB^ + BC^ =\^ +\^ =2, 

i.e. AC has a length v2 or has no length at all. If we accept that every segment 

has a length, we must find a way to include v2 in our number system. These new 
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objects will be called irrational numbers. The set of all the rational and the 
irrational numbers, is called the real number system and is denoted by 9t . 




Fig. 2.4.1. Introducing V 2 , 



Accepting the 'existence' of irrational numbers is only one step. We still have 
to formally define the real numbers, define arithmetic operations between them 
and show that the outcome 9t is indeed an extension of R . The popular way for 
doing it, is by using the Dedekind cuts. 

Definition 2.4.1. A Dedekind cut is an ordered pair {A,B) of sets of rational 
numbers, which satisfy the following requirements: 

1. A and B are nonempty. 

2. Every re R belongs to either A or B . 

3. Every ae A is less than every be B . 

For example, A may consist of all the rational numbers less than or equal to 3, 
while B consists of all the rational numbers that are greater than 3. There are four 
possible types of Dedekind cuts (or simply 'cuts'): 

(a) A has a largest number and B does not have a smallest number. 

(b) B has a smallest number and A does not have a largest number. 

(c) A has a largest number and B has a smallest number. 

(d) A does not have a largest number and B does not have a smallest number. 

We can immediately dismiss one type. Clearly, case (c) is impossible: If a is the 
largest number in A and b is the smallest number in B , then (a + b)/2 is a 
rational number greater than a , smaller than b , and thus cannot belong to either 
A or B . This contradicts part (2) of Definition 2.4.1. Consequently, a cut of type 
(c) does not exist. We can also disregard cuts of type (b). Indeed, a cut (A,Z?)of 
type (b) with a smallest be B , is merely an alternative representation of the cut 
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(A up},B -pj) of type (a). Thus, we should consider only two types of cuts: (a) 

and (d). If a cut is of type (a) it is called a rational cut. A type (d) cut is called an 
irrational cut. There is a simple one-to-one correspondence between all the cuts of 
type (a) and all the rational numbers re R , namely 

r ^(A,.,B,); A, = {s \ s e R, s < r} , B^ = {s \ s e R, s > r} (2.4.1) 

The 'other' cuts, if they exist, are the new 'irrational numbers'. Our first objective 
is to define equality and arithmetic operations among the Dedekind cuts. 

Definition 2.4.2. Two arbitrary cuts (A,Z?) and (C,D) are called equal if and 
only if A = C and B = D. 

Definition 2.4.3. The i'Mm of arbitrary cuts (A,Z?) and (C,D) is the cut 

(E,F) = (A,B) + (C,D) 
where E = {e\e = a + c; ae A,ce C} and F ={f \ f =b + d \be B ,de d}. 
Showing that (E, F) is a cut, is left as an exercise for the reader. 

Definition 2.4.4. The multiplication of arbitrary cuts (A, B) and (C, D) is the cut 

(E,F) = (A,B)(C,D) 

where the sets E,F are determined as follows: 

(1) If A,C both have nonnegative rationals then 

F={f\f=bd;beB,deD} , E = R-F 

(2) If A has nonnegative rationals and C has only negative rationals 

E = {e\e = bc;be B,ceC} , F = R-E 

(3) If A has only negative rationals and C has nonnegative rationals 

E = {e\e = ad ■,ae A,de d] , F = R-E 

(4) If A, C both have only negative rationals 

F={f\f=ac;aeA,ceC} , E = R-F 
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and if F has a smallest rational we transfer it to E and maintain a cut of type (a). 
Showing that {E, F) is a cut (of type (a) or (d)), is left as an exercise for the 
reader. 

Given a pair of arbitrary cuts (A, B) , (C, D) , we can show that the equation 

(A,B) + (X,Y) = (C,D) 

has a unique solution {X,Y) which we denote by (A,B)-(C,D) (subtraction). 
We also have a unique solution to 

(A,B)(X,Y) = (C,D) 

provided that (A,B) i^ . This solution is denoted by (C,D)I{A,B) {division). 
We will now define order among the real numbers. 

Definition 2.4.5. A cut (A, B) is greater than a cut (C, D) if and only if there is a 
rational number r such that r&A,riC. If there is a rational number r such 
that r&C,riA we say that (A,B) is less than (C,D) . 

The cut {A,B) where A contains the rational and all the negative rationals, 
is the real number 0. A real number is positive, if it is greater than 0. It is negative, 
if it less than 0. We denote by JJ""" , 5{" , the sets of the all the positive and the 
negative real numbers, respectively. Now, consider two positive real numbers a,b 
such that < a < i) . What can be said about the sums a + a,a + a + a,...,na,...l 
It was Archimedes who first realized that no matter how small a is, we can 
always find some positive integer n (possibly very large) for which na>b . 

Lemma 2.4.1. (Archimedes' axiom) For arbitrary positive real numbers a,b , 
there is a positive integer n such that na > b (This lemma is valid also for 
arbitrary a, be 3{). 

Proof. 

Assume that a,b are rational numbers. Let a = r/s , b = p/q where r,s,p,q are 
positive integers. In order to get na> b , we should find n such that 
n{r/s) > (p/q) , or n > (sp/rq) . Thus, we may choose n = sp . The proof for 
positive real numbers a,b (which are now represented by cuts) is similar and is 
left as an exercise for the reader. 

Corollary 2.4.1. For every arbitrarily small e >0 there is a positive integer n 
such that l/n < e (substitute a = e , b = 1 in Lemma 2.4. 1). 
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It can be seen that under the one-to-one correspondence of Eq. (2.4.1), 9t is an 
extension of R . We will now show the existence of irrational cuts, i.e. of 
'irrational numbers'. Consider the cut a = {A,B) where A consists of all the 
rationals whose square is less than 2, and B - of all the rationals whose square is 
greater than 2. Since there is no rational number x such that x = 2 , every rational 
is in either A or B . Assume that there is a largest rational ae A. . If we define 

e = min(fl, ) 

then f is a positive rational which satisfies a <(a + e) < 2 i.e. (a + f) e A 
contradicting our assumption. Similarly, there is no smallest rational in B . Hence, 
{A,B) is an irrational cut. Thus, the set of irrational cuts is nonempty! We will 

now show that this particular cut solves the equation x =2 . 

Lemma 2.4.2. Let r, s denote rational numbers such that < r < s . Then a 

2 

rational number t : r <t < s can be found. 

Proof. 

We should consider the following three cases: 

(a) Let r < s <1. We wish to find a positive rational number m/n , m <n such 
that: 

n\ "^^ ^ r'i\ (in + if -^ 

(1) ^-<'5 ' (2) 2 -^ 

n n 

For any prefixed n , we can use Lemma 2.4.1 and find a nonnegative integer m 
that satisfies (2). Let Mq be the smallest m for which (2) holds. Then, Mq and n 
satisfy (1) as well. Since 5 < 1 we also must have Mq <n . We will now show 
how to prefix n so that 



r<— ^ 
n 



Indeed 



OTq (mq +1) in^ _ IniQ +1 3« _ 3 

2 - 2 2~~ 2 ^ ~^~~ 

n n n n n n 
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and since by corollary 2.4.1 we can find n = nQ that satisfies O/hq) < s - r , we 
obtain 



nil 






and finally r <mQ /iiq . We therefore choose t = mQ /hq and the proof is 
concluded. 

(b) Let r <l< s . Here we simply choose t = 1 . 

(c) Let l< r < s . This case is left for the reader (transfer to case (a)). 

Theorem 2.4.2. The previously defined irrational cut a satisfies a =2 . 
Proof. 

Consider the cut J3 = (C, D) = a . Every element se D , can be written as 

s = TjTj , r[,r2 e B . Since rj > 2 , rj > 2 we also have s >2 . The opposite is 
also true: given a rational number r > 2 , we can apply Lemma 2.4.2 and find a 

2 2 

rational number t such that 2 < f < r . Consequently teB and t e D . 
Therefore re D as well and we conclude that D consists exactly of all the 
rationals which are greater than 2. Thus, 2e C and (C, D) = 2 . 

Solving the equation x = 2 is a particular case of a more general problem: 
Given ae R^ , ne N , find a positive real number a e St""" , such that a" =a . It 
can be shown that this problem has a unique solution (see problem 10 below) 
which we denote by ija or a''" . We can now go further and define; 

a'^l" =\l(al"), aeR'^,neN (2.4.2) 

a""!" =4^ , aeR\neN ,mel 



Finding a solution to x" = a can be done by using Newton's Binomial 
Theorem or even just Bernoulli's inequality (problem 13 below) combined with a 
process similar to that applied for solving x = 2 . We will present the binomial 
theorem - a powerful mathematical tool, prove it, but leave the rest of solving the 

equation x" = a to the interested reader. We start by defining the binomial 
coefficients. 
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Definition 2.4.6. For arbitrary nonnegative integers n, k sucii tiiat k <n , tiie 

n 1 n\ 



number 



^k) k\(n-ky. 
defined as 1). 



is called the binomial coefficient n over k (0! is 



Lemma 2.4.3. The binomial coefficients satisfy 

n] (n + \ 



k-\\ \k 



The proof, based on Definition 2.4.6, is straightforward and is left for the reader. 
We now define the power notation and use it combined with mathematical 
induction to get the binomial theorem. 

Definition 2.4.7. For arbitrary real number a and positive integer n : 



a = a , a""*" = a" ■ a 



Theorem 2.4.3 (Newton's Binomial Theorem). For arbitrary a,b e 9t , « e N : 



(a + /7)" = fl" + 1 " |a"-'Z7 + 1 " |a"-2^2 ^ . . . + 



n-2 



.2,«-2 



n-\ 



k=Q 



t-y^ 



ab"-'+b" 



(2.4.3) 



Proof. 

(a) M = 1 . Both sides of Eq. (2.4.3) are a+b . 

(b) Assume Eq. (2.4.3) to hold for arbitrary n > 1 . Then 



(a + b)"^^ = (a + b)" (a + b): 



k=0 



z:k-'*' 



(a+b) 



n + \ 



>il 



a +\ \a b + \ \a b + ...+ 



n-\ 



la b" +\ lab" 



a"b + 



a b +...+ 



n-2 



: 2-1 n-\ , 
a b + 



M-l U + 1 
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By adding the two last rows and applying Lemma 2.4.3 obtain that Eq. (2.4.3) 
holds for M + 1 as well, which concludes the proof. 

At this point, it should be emphasized, that during actual calculations, we do 
not have to present each number as a cut with two infinite sets of rational numbers 
and perform our calculations on the elements of these sets. This would be correct, 
but endless, tiresome and unnecessary. Instead, we represent each number in the 
familiar form of a decimal fraction dQ.dyd2... , finite or infinite, and use all the 
usual standard techniques to calculate. If the number is rational, the equivalent 
decimal fraction is either finite, or infinite and periodic. For example, 0.25 
represents the rational number 1/4 while 3/11 is represented by 0.2727... . An 
irrational number is represented by an infinite non-periodic decimal fraction: 

v3 =1.7320508 Note that a decimal fraction dQ.d^d2..- , which represents a 

cut {A,B), actually defines the set A . Indeed, A is the union of all the sets 

£), ={r\re R , r <dQ.did2...di} . 

The set of the real numbers can be also represented geometrically, as all the 
points of a straight line called the real line or the real axis (Figure 2.4.2). The one- 
to-one correspondence between numbers and points defines the order of the real 
numbers from left to right. 

I I I \ \ \ \ 

-4.5 -2.2 -V2 0.5 2 ^ 

Fig. 2.4.2. The real line. 

Those readers, who feel somewhat uncomfortable with the Dedekind's cuts and 
with the irrational numbers, being located on the real line, may try and visualize a 
real number as a point where the two disjoint sets A and B meet. This meeting 
point, is the rightmost point of A , if the number is rational. If A has no rightmost 
point, A and B still meet at some point, which now represents an irrational 
number a (Figure 2.4.3). 



A « B 



Fig. 2.4.3. An irrational number: A and B meet at 01 . 

PROBLEMS 

1 . If ^ is a prime number, then x =k has no rational solution for any positive 



integer / > 1 . 
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2. Let a and r be irrational and rational numbers respectively. Then, the real 
numbers a + r , «r are both irrational. 

3. Find the solution of (A,5) + (X,y) = (C,Z)) for arbitrary cuts 
(A,B),(C,£)). 

4. Find the solution of (A,B) •(X,y) = (C,Z)) , (A,5) ^^ for arbitrary cuts 
(A,B),(C,D). 

5. Let a,l3,y denote arbitrary cuts. Show: (a) afi = fia (b) {ali)Y = aifiy) . 

6. Given rational numbers r,s \r <s find an irrational number a : r <a < s . 
(Hint: Find a rational number t between r and s ; then use Archimedes' 
law and obtain an irrational number of the form / + v2/« that does not 
exceed s). 

7. Given a real number a = (A,B) , show: (a) r <a for every rE A . (b) a < r 
for every re B . 

8. Given irrational numbers a,/^ ; a < j8 find a rational number t .a <t < P . 

9. Given real numbers (rational or irrational) a,P ;a < P find a rational number 
t .a <t < P and an irrational number y.a<y<p. 

10. If a,/5 are positive real numbers such that a = /? = 2 , then a = fi . 

11. \f a,/5,y are real numbers such that a> P ,y>Q , then ay > ^5/ . 

12. If a,y5 are real numbers such that a> ji , then a+ y> P+ y for all real / . 

13. (a) Show a >0 for arbitrary real number a. (b) Prove Bernoulli's 
inequality: (1 + a)" >\ + na for all positive number n and real number a 
such that a>-\ . 

14. Prove Lemma 2.4.3. 

15. Show the existence of a solution to x" = a for arbitrary a e R^ , ne N. 



16. Show that for arbitrary « e A^ : V =2" 



..r^ 



2.5 Additional Properties of the Real Numbers 

An important notation associated with numbers is the absolute value. We define it, 
derive the triangle inequality and then use it to get other basic relations, such as 
the Cauchy - Schwarz inequality. 

Definition 2.5.1. For every real number a e 9? , we define the absolute value of 



a , fl > 

(2.5.1) 
-a , fl < 
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For example, |l7| = 17 , |-13|= 13 , |0| = 0. 

Some basic properties of the || operator, which can be easily derived, are 

(1) -Ifll < fl < Ifll , ae 5R. 

(2) [flZ?! = |fl||fe| , a,be3{. 

(3) \a/b\ = \a\/\b\,a,be3i,b^0. 

Less trivial is the next result. 
Theorem 2.5.1. The triangle inequality 

\a + b\<\a\+\b\ (2.5.2) 

is valid for arbitrary real numbers a, be 3i . 

Proof. 

Six cases should be considered: 

(a) a>0 ,b>0: \a+b\ =la\ + \bl =\a\ + \b\ . 

(b) a>0,b<0,a + b>0: \a + b\ = a + b < a - b = \a\ + \b\ . 

(c) a>0,b<0,a+b<0: \a + b\ = -a - b < a - b = \a\ + \b\ . 

If change the roles of a and b , we obtain the three remaining cases. 
By using induction we derive the more general triangle inequality 

fl[ + flj +••• + <3n - ri "*" r2 "''•••"'' r« 

The next result, known as the Cauchy - Schwarz inequality, is most helpful in 
numerous applications in analysis. It is based on the following lemma. 

Lemma 2.5.1. If the inequality ax +bx + c>0 holds for some fixed a,b,ce 3i 
and all x e 9t , then b^ < Aac . 

Proof. 

If fl = , then ax +fox + c>0,xe9t implies i>x + c>0,x69?. Consequently 
b = (why?) and b < Aac since both sides vanish. If a > we rewrite 
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2 b 2 b 

ax +bx + c = a(x-\ ) +c >0 

2a 4a 

and since the inequality must hold for all x e 5R , it is valid also for x = -b/{2a) 

which implies c-b^/(4a)>0 , i.e. (4ac-b^)/i4a)>0 . Since a>0 the 

numerator is nonnegative and we get 4ac > b . The case a < cannot occur 
(why?). This concludes the proof. 

We now introduce the sigma notation, defined as 



n 

^a^ =a 



i+flj+.-. + fli 



k=l 



Theorem 2.5.2 (Cauchy - Schwarz inequality). Let ne N and a^,b^e3l for 
l< i <n . Then 



\k=i J 



n \ n 



Yj'^kbk ^ X"* Z^' 



2 
'k 
V k=\ J\ k=l J 



(2.5.3) 



Proof. 



Define f ix) = ^(a,^+bi,xf ,xe%. Then f(x)>Q,xe%,i.e. 



k=\ 



( « ^ 




( " ) 


Yj-i 


x^+2 


^a^b,^ 


\k=\ ) 




U=i J 



k=\ 



and by applying Lemma 2.5.1 we conclude Eq. (2.5.3). 

In the particular case b^. =\ ,\<k <n , the Cauchy-Schwarz inequality implies 



f n \' 



^fl^ <M^a 



V*=i ) 



k=l 



or 



in \ 

k 



k=l 



n 



k=\ 



(2.5.4) 
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Definition 2.5.2. For arbitrary positive real numbers a^. ,l<k<n we define 



a, + a, + ... + a 
A(a^,a2,...,a„) = — — (arithmetic mean) (2.5.5) 



G{ai,a2,...,a„) = ^Jala2■■.a,^ (geometric mean) (2.5.6) 

H(a^,a2,...,a^) = — y ^ (harmonic mean) (2.5.7) 




2 2 2 

fli + fl9 + . . . + a 



Q(a^,a2,...,a.^) = J — — (qudratic mean) (2.5.8) 

V n 

By virtue of Eq. (2.5.4) we have A(ai,a2,...,a^) <Q(ai,a2,...,a^) , i.e., the 
arithmetic mean never exceeds the quadratic mean. For example 



1 + 2 + 6 (1^+2^+6^ (41 
< 



3 V 3 V 3 

We will now show that the geometric mean never exceeds the arithmetic mean. 

Lemma 2.5.2. For arbitrary positive real numbers a,b e 9^ , a < i> and me N we 
have 

' , \ m+l 



m + l 
Proof. 

By virtue of Bernoulli's inequality we get 

ma + bY^ ( b-aY^ m+i( , ^ b-a ^ ^+il (m + l)(b-a) . ^ 
a-\ =a IH >a \l-\ \ = a b 



m + \ ) \ m + l J y a(m + \) J [^ a(m + l) 

which concludes the proof. 
Theorem 2.5.3. For arbitrary positive real numbers a^ ,1< k < n we have 

G(ai,a2,...,fl„) < A(fl[,fl2,---,fl„) (2.5.9) 
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Proof. 

Without any loss of generality, we may assume that a^, ,l<k<n are already 
given in an increasing order, i.e., flj < 02 S...^a„. Replace 0^,02 by the two 
equal numbers 0^2 = ("^i +'^2)/^ • By applying Lemma 2.5.2 with m — \ we 
obtain a new set of increasing numbers a2,CC2,aT,,---,an with the original 
arithmetic mean, but with a non-decreasing geometric mean. In the next step we 
replace or2 ' ^2 ' "^3 ^y '^he three equal numbers a^ = (2«2 + «3 ) and leave the 
remaining numbers unchanged. By using Lemma 2.5.2 with m — 2 ws get the 
increasing numbers «3,«3,«3,a4,...,a„ still maintaining the original arithmetic 
mean and a non-decreasing geometric mean. After (n - 1) steps we obtain n 

n 

equal numbers a„,a„,...,a„ (a„ = /^a^ ). Denote the original arithmetic and 

k=\ 

geometric means by A,G and the final means by A' ,G* respectively. Then, 
A" = A = G = «„ ,G< G which implies G < A as stated. 

It can be easily derived that G{ai,a2,...,a^) = A(ai,a2,...,a^) if and only if all 
the numbers are equal. Also, by applying Theorem 2.5.3 on the inverses of 
ai^,l<k<n, i.e., to l/ai,l/a2,---,l/fl„ we get 

H{a^,a2,...,a^)< G(a^,a2,...,a^) . Hence, for any positive real numbers, we have 

H(ai,a2,...,a^) < G(ai,a2,...,a^,) < A(fli,a2,---,fl„) ^ 2('2i''32,...,fl„) (2.5.10) 

Example 2.5.1. Consider the numbers 1,3,4,7,10 . The calculated means 

are A = 5 , G = ^840 » 3.845 , g = V35 » 5.916 , // » 2.738 and they satisfy Eq. 
(2.5.9) (the symbol = means 'approximately'). 

PROBLEMS 

1 . For arbitrary real numbers a,be3i: \a-b\> U - \bn . 

2. For arbitrary real numbers a,b,ce3i: \a - b\<\a - c\ + \c - b\ . 



3. For arbitrary a, be 3^: \ab\ = \a\\b\ and if b ^0: 



4. If a < c < b for a,b,ce 9t , then la-Z?! = la -cl + lc-Z?! . 

5. Find all X for which Ix + 2| < 3 (Hint: assume first x + 2 > , then x + 2 < ). 
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6. Find all X which satisfy Ix + 5| < Ix - 7| . 

7. Show a + ->2 for all St"" . 

a 

8. Show a>b <^ a^ >b^ for all a,be3{. 

9. Prove the Minkowsky Inequality 



f n 






v'^ ( n \y 



\k=l J 



Z«' + Z^' 



f n ^1/2 

2 



for arbitrary real numbers ai^,bi^ ,l<k<n (Hint: use the result of problem 6 
combined with the Cauchy-Schwarz inequality). 



10. Show n\< 



n + 1 



ne N without using induction. 



11. Show r 



(n i)/2 ^ ^ ne A? for all re R'^, r^l without induction. 

n(r — l) 



12. Let the set {fl[,fl2,- ••,«„} satisfy a, ^ , 1 < / < « . Show: 



"^^S^'^S 



1 



13. Use the result of problem 12 to prove: n < l-\ I-...H — |,«>1. 

2 12 n. 
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In this chapter we define the notations of function and sequence and introduce the 
most important concept of calculus: the limit. This chapter is vital to the 
understanding of any further reading; in particular, the reader must come to 
control the subject of limit and convergence. 

3.1 Introduction 

A preliminary notation, necessary for introducing the function concept, is the 
Cartesian product. 

Definition 3.1.1. The Cartesian product of given sets A and B is the set AxB 
of all the ordered pairs {a,b) , where ae A and be B : 

AxB = {(a,b)\ae A,be b} (3.1.1) 

Example 3.1.1. Let A = {1,2,4} , fi = {2,3} . Then 

A X B = { (1, 2), (1, 3), (2, 2), (2, 3), (4, 2), (4, 3) } 

Definition 3.1.2. Any subset of the AxZ? is called a relation from A to B . The 
set of all the ordered pairs of a relation present the graph or the curve of the 
relation. 

Example 3.1.2. In the previous example, the set {(1,2), (2, 2), (2, 3), (4, 3)} is a 
relation from A to B . In order to obtain the graph associated with this relation, 
we first draw two infinitely long straight lines, perpendicular to each other. These 
horizontal and vertical lines are called axes and are usually denoted by x and y 
respectively. Secondly, we mark all the points corresponding to the ordered pairs, 
relative to the intersection point of the axes (Fig. 3.1.1), called origin and defined 
as (0,0). 
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Fig. 3.1.1. The grapii of the relation in Example 3. 1 .2. 
Example 3.1.3. The following sets are relations from 9t to 9t : 

(a) S^={{x,y)\x^^,y = r' Qa) S^={{x,y)\x,y^^ ,x^ + y'^ =1} 

In case (a) we calculate x for every x e 91 and place the ordered pair (x, x ) in 
5i . In case (b), given an arbitrary xe9t, we find all the y 's such that 

2 2 

X + y =1 and place each (x, 3^) in ^2 . 

Throughout this text we will usually discuss relations where A and B are 
subsets of 9t . The most commonly used subsets are the intervals, which are 
defined next. 

Definition 3.1.3. The following subsets of 9t are called intervals: 

\a,h\ = {x\a<x<h , a,b e 9t} (closed interval with endpoints a and b ) 
{a,b) = {x\ a < x<b , a,b e 3i} (open interval with endpoints a and b ) 
[a,b) = {x\ a < X < b , a,b e 9t 



, 1 r (semi open interval with endpoints a and b ) 

{a,b\ = {x\a < x<b , a,b e 9t 

A semi open interval is also called 'semi closed' or 'half open (closed)'. The 
symbol ' = ' means "defined as". 

For example, the set of all the real numbers between 2 and 17.5 (including 2 
and 17.5) is the closed interval [2,17.5j. We often rewrite the set of all the real 
numbers, 9t , as 9^ = {x I -0° < x < 00} or as 3i = (-00,00) . By (-00, «) we mean 
"all the real numbers less than a". By (b,°°) we mean "all the real numbers 
greater than b ". If the endpoint is included we write (-00, «] and [^7,00) . Note 
that the symbols -00 (minus infinity) and 00 (infinity) are not legitimate real 
numbers. 
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Example 3.1.4. All the real numbers greater or equal to 5 are denoted by [5,oo) . 

All the real numbers less than v2 , are represented by (-oo,v2) . The set of all the 
real numbers can be rewritten as the union (-oo,3) u [- 3,10ju (10,oo) . 

PROBLEMS 

1. Let A = {x,l},B = [2, a, 0} .Find the Cartesian products AxB and BxB. 

2. Let A = {x,y,l} , B = {2,a,0} ,C = {2,b,y} . Find Ax{B-C) and 
(A-C)xB . 

3. Let A,B denote arbitrary sets. Show that AxB = BxA, if and only if 

A^B. 

4. What is the relation S ={(x,y) I xe {2,3} , -oo < 3; < oo} ? 

5. Consider the relation S ={(p,q)\ p,q,(p + q + i) primes} . Find the part of the 
graph of S for which 1 < p,q < 10 . 

6. Define a relation whose graph is the upper positive half of the x- y plane. 

3.2 Functions 

We are particularly interested in one type of relations, called 'functions'. 

Definition 3.2.1. A function / , is a relation from A to B , whose graph, denoted 
by graph(f) , satisfies: If (a,b),(a,c) g graph(f) then b = c . The set 

Df ={alflE A and there exists be B such that (a,b)e graph(f)} (3.2.1) 

is called the domain of / . The set 

Rf ={b\be B and there exists ae Df such that {a,b)e graph(f)} (3.2.2) 

is called the range of / . 

The most important feature of a function / is that for every xe Df , there 
exists a unique yeB, such that (x,y)e graph(f) . We call y the image of the 
source X and denote it by /(x) . If we restrict ourselves to sets A and B that are 
subsets of 3i , the function / is called a real-valued function. Note that / is 
always single-valued. We also say that / is a mapping of Df^A to (or 

into) B , 01 a mapping from A to B . The first coordinate x of an ordered 
pair(x, 3')e graph(f) , is called the independent coordinate. The second one, y , 
is the dependent coordinate. 
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Example 3.2.1. The relation {(x,y)\0 < x <l , y = x + x} is a function from 9t 
to 9t . It is also a mapping of [0,lj to 3i . Indeed, for every X between and 1, 
the image y is uniquely determined. On the other hand, the relation 



S = {(x,y)\ x,ye3i , X +y =l}is not 



function, since for arbitrary 



X, < X < 1 there is more than one ordered pair (x, y) , which belongs to the graph 
of S . In fact, there are two different y^ and j'j ' such that {x,yi),(x,y2)are in 
gmph(S) , i.e. x + y^ = x + y2 =1 ■ For example x = 1/2 has two images 
}'i = v3/2 and y2=-yl3/2. For an arbitrary 

y- 



we can represent S by 



il 



:+Vl 



an explicit form of the two images y^ = +Vl 



and 



ji = -Vl - X . Usually, by writing ^(O we mean the unique positive square root 
which is now a function and not just a relation. The graphs of the function 
y = X + X and of the relation S are given in Fig. 3.2. 1 . 



X +y =1, 




Fig. 3.2.1. A function (solid) vs. a relation (dashed). 



Whether a given relation 5 , is a function, can be determined by the following 
test: If every vertical line through {x,0) , xe D^ (the relation's domain), 
intersects the relation' s graph at exactly one point, then this relation is a function. 
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Example 3.2.2. In the previous example, the function y = x + x has the domain 
[O, ij and we can easily derive that its range is [0, 2j . However, if we agree to 

extend the domain to the largest subset of 9? , over which the expression x + x is 
well defined, i.e., has a single value, then the domain is clearly 3i itself. In order 
to decide whether an arbitrary number a is inside the range, we need to solve the 

equation x + x = a whose formal solutions are given by 

_ - 1 + Vl + 4fl 

Xi 9 — 



Since there is no real number, which is the square root of another negative real 
number, we must have 1 + 4fl > . Thus, the maximum range of the function, 
defined as the image of the maximum domain, is the interval [-1/4, oo) . 

Consider a function / from A to B , denoted by f : A^> B , and an arbitrary 
number xe5{. If xeDr, we say that / is defined at x and f{x) exists. 
Otherwise, if xi Df , then / is not defined at x and f{x) does not exist. It is 

usually relevant to consider only the domain of a function rather then whole set 
A . Therefore, by writing f : A^> B we assume (unless otherwise stated) A = Df 

and Rf c B c 91 (it may be not be trivial to obtain the exact Re ). For example, if 

2 1 

f(x) = l + x H y then /:9t— ^9{ although a further inquiry reveals 

l + x 

R^ =[2,oo). 

A function f : A^> B which satisfies x^ it X2=> fix^) ^ /(X2) for arbitrary 
different Xj , Xj e A , is called one-to-one (also: 1 - 1 ). If /?y = B , we say that / 
maps A onto B . 

Example 3.2.3. Consider the function y = x + x from 9? to 9? . It is not one-to- 
one since for example Xj = 3 and X2 = -4 provide the same image y = \2 . It is 
not onto, since the real number -10 is not within the function's range (see 
Example 3.2.2). On the other hand the function y = x +1 from 9t to 9t is both 
one-to-one and onto. 

Every one-to-one function f '■ Dt -^ Rf , can be associated with another one-to- 
one function g :Rf ^> Df , defined as g(y) = x , ye Rf , where x is the unique 
source of y in Df . Thus, D = Rf and R = Df . The function g is called the 
inverse of / and is denoted by /" . For example, if y = f(x) = 5x + l, the 
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inverse function is x = g(y) = . In order to maintain consistency we will 

keep using x as the independent coordinate and write g = f^ ={x- 1)/5 . 

We now present the concepts of bounded and monotone functions. First, we 
define a bounded set. 

Definition 3.2.2. A set A c 91 is called bounded, if there is a real number M > 
such that 

\x\<M , xe A (3.2.3) 

If there is no such M , the set A is called unbounded. If there is a real number 
M[ such that x < Mj , xe A , we say that A is bounded above and Mj is an 

upper bound of A . If there is a real number M2 such that M2 < x , x e A , we 

say that A is bounded below and Mj is a lower bound of A . 

It is easily seen that a set is bounded, if and only if, it is bounded both above and 
below. Thus, the information that a set A is unbounded, implies that A is at 
least unbounded above, or unbounded below. 

Definition 3.2.3. Let A be a bounded above set. If a smallest upper bound of A 
exists, we say that A has a supremum or a least upper bound and denote it by 
either sup( A) or l.u.b.{A) . If A is bounded below and a largest lower bound of A 
exists, we say that A has an infimum or a greatest lower bound and denote it by 
either inf( A) or g.lb.(A) . 

Example 3.2.4. The set 9^""" = (0,oo) is bounded below and has the infimum 0. The 
set [2,7) is bounded. It has the supremum 7 and the infimum 2. The set of the 
natural numbers is unbounded below and unbounded above. 

The next result confirms the existence of sup(A) for an arbitrary bounded above 

set. 

Theorem 3.2.1. If A is a nonempty bounded above set, then sup(A) exists. 

Proof. 

Let B consist of all the rational numbers, which are upper bounds to A and let 
C = R-B . We will show that the pair {C,B) is a Dedekind cut. Clearly we 
constructed this pair so that every rational number is in one of its sets. Since A is 
bounded above, it must have at least one upper limit, say Mj . Find a rational 
number r : Mj < r < Mj + 1 . Thus reB, i.e. B is nonempty. The requirement 
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that A itself is nonempty guarantees the existence of some ae A. Let r be a 
rational number between a and a-\ such that a-\<r <a . Thus, r is not an 
upper bound of A , i.e. ri B and consequently re C which confirms that C is 
also nonempty. It remains to show that every number in C is smaller than every 
number in B . Indeed, let rEC,seB. Since r is not an upper bound of A , 
there is a number ae A such that r < a . Since a < s we get r < s . Hence 
a = {C,B) is a Dedekind cut. 

We now prove that a = sup(A) and start by showing that a is an upper bound 
of A . Indeed, if this is false, then for some a e A we have a <a . Find a rational 
number r.a<r<a. Since a < r we must have re B , i.e. r is a rational upper 
bound of A which contradicts r < a . Next, we show that no upper bound of A , 
is smaller than a . Otherwise, let P <a be an upper bound of A . Find a rational 
number r : ji <r <a . On one hand r <a , i.e. reC. Consequently, r is not an 
upper bound of A . On the other hand, j3 <r and since /^ is an upper bound of 
A , so is r . This contradicts the previous assumption, i.e. a = sup(A) . 

The proof of the next result is left for the reader. 

Theorem 3.2.2. If A is a nonempty bounded below set, then inf(A) exists. 

Corollary 3.2.1. Every nonempty bounded set A has both a supremum and an 
infimum. 

If the number a = sup(A) belongs to A , it is the maximum number in A and we 
denote a = max(A) . If the number a = inf(A) belongs to A , it is the minimum 
number in A and we denote a = min(A) . For example, the set A = [2,7] satisfies 
sup(A) = 7 , inf(A) = min(A) = 2 . This particular set has no maximum. 

We now define bounded and unbounded functions. 

Definition 3.2.4. A function / : A ^ fi , A, 5 c 9? is called bounded, if there is a 
real number M > such that 

\f(x)\<M , xe A (3.2.4) 

If there is no such M , we say that / is unbounded. If there is a real number Mj 
such that /(x) < Mj , xe A, we say that / is bounded above and Mj is an 
upper bound of / . If there is a real number Mj such that Mj < f{x) , xe A, 
we say that / is bounded below and M2 is a lower bound of /. 

For arbitrary function / , we can apply Theorems 3.2.1-2 to the set Re and 
conclude that a bounded above / has a smallest upper bound, and a bounded 
below / has a greatest lower bound. 
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Definition 3.2.5. Let / be a bounded above function. The smallest upper bound 
of / is called supremum or least upper bound and is denoted by either sup(/) or 
l.u.b.if) • If / is bounded below, the largest lower bound of / is called infimum 
or greatest lower bound and is denoted by either inf(/) or g.l.b.{f) . If sup(/) 
belongs to Rt , it is the maximum of / and we write sup(/) = max(/) . If inf(/) 
belongs to Rf , it is the minimum of / and we write inf(/) = min(/) . 



whose domain is 5R . Since 



Example 3.2.5. Consider f{x) = - 

\ + X 

\x\ < |2x| <\ + X ,xE9t, / is bounded by 1 . Further investigation provides 

sup(/) = 0.5 and inf(/) = -0.5 . Since /(I) = 0.5 and /(-I) = -0.5 we can also 

write max(/) = 0.5 and min(/) = -0.5 . 



Example 3.2.6. Let f{x) 



1 



Y + x 



5R and x^-\ (Fig. 3.2.2). We will show 



that / is unbounded above. If this is false, there is M > such that 



1 



l + x 
1 



■<M 



9t . Assume x>-l. Then, 0< 



1 



l + x 



< M and consequently 



< 1 + X . However, the last inequality fails to hold for x = -\ + 2/M . Hence, 

f{x) is not bounded above. It can be similarly shown that it is not bounded below 
as well. However, if we restrict the function's domain to [-0.8,0°), it becomes 
bounded, with sup(/) = 5 , inf(/) = . 




Fig. 3.2.2. A function unbounded above and below. 
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Example 3.2.7. The function /(x) = x - Ixl , x e 3i which satisfies 

0,x>0 



f(x)- , 

[2x , X < 

is bounded above by but is not bounded below. Hence, / is unbounded. The 
function /(x) = -^ , x ^ is bounded below by but unbounded above. It is 

X 

therefore unbounded. 

The next notation is 'monotone functions'. 

Definition 3.2.6. A function / is said to be increasing if Xj < X2 => f{xi) < /(X2) 
for all X, ,X2 e Dr . If Xj < X2 ^> /(xj) > f{x2) for all Xj,X2 e £)y , the function is 
called decreasing. If xj < X2 ^> /(xj) < /(X2) for all X],X2 e Df , the function is 
called strictly increasing and if Xj < X2 => /(xj) > /(X2) for all Xi,X2 e £)r , the 
function is strictly decreasing. A function / which is either increasing over Df or 
decreasing over Df , is called monotone. 

Example 3.2.7. The function /j (x) = [xJ , x e 5R defined as the greatest integer less 
than or equal to x , is increasing over 3i , but not strictly increasing over any 
interval (Figure 3.2.3).The function /; (x) = x - Ixl , x e 5R is also increasing but not 
strictly increasing. If we restrict it to x < it is strictly increasing. The function 

/2 (x) = -Y , X ^ is not monotone throughout its whole domain. However, it is 

X 

clearly strictly increasing over (-co,0) and strictly decreasing over (0,00). 



Fig. 3.2.3. Graph of y = [x] . 
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Example 3.2.8. Sometimes, finding the sub-domains where a function is either 
increasing or decreasing, is less simple than in the previous example. Consider the 



function /j (x) : 



-3x + l 



x^+1 



, xe 5R whose graph in Fig. 3.2.4 represents three 



sub-domains of monotony: 



1. -oo < X < -1 : strictly increasing. 

2. -1 < X < 1 : strictly decreasing. 

3. 1 < X < oo : strictly increasing. 

Finding these domains using elementary algebra, is possible but somewhat 
tiresome. Later, after introducing differentiation, we will familiarize ourselves 
with a more elegant procedure for determining the sub-domains where an arbitrary 
function is increasing or decreasing. 




Fig. 3.2.4. Graph of (x^ - 3x + l)/(x^ + 1) . 

We now define the basic arithmetic operations between functions. 

Definition 3.2.7. Let / and g denote arbitrary functions defined over the same 
domain D . We define 



(f + g){x) = f{x) + g(x),xeD (sum) 
(/ ~ 8)ix) = f(x)- g(x) , xe D (difference) 
(.fg)ix) = fix)g(x) , xe D (multiplication) 
and if g(x) ^ , xe D 



(3.2.5) 
(3.2.6) 
(3.2.7) 
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L \x) = iM ^ xeD (division) (3.2.8) 

8) six) 

Example 3.2.9. Let /(x) = 3x^ - Vx , x> and g(x) = l-x^ xe 5R . Then 

r r\ -x ^ _ r 

(/ + g)(x) = 4x^-V^ + l,x>0 and M- {x)=— f^,x>0,x^l. 

\g 1-x^ 



Definition 3.2.8. Let / and g denote arbitrary functions such that R is a 
subset of Df . The function (/ o g)(x) = f{g{x)) , xe D is called the composite 
function of / and g . 

Example 3.2.10. If f(x) = x^ , g{x) = ^lx + [ then (/ o g)(x) = (-Jx + lf =x + l. 
Note that every image of g is in Df=3i as requested. The domain of / o g is 
Dg=lx\x>-1]. 

PROBLEMS 

1. Find the domains of the following relations from 9t to 51 and determine 
which of them are functions. 



x+ y 



(a) {(x,y)\x^ + y*=l} (b) { (x, y) I y = V^^ j (c) \(x,y) 



2x- y 
(d) {ix,y)\^-y^=x 

2. Find the domain and the range of /(x) = r- and draw its graph. 

(1 + x) 

3. Find the domain and the range of /(x) = —r and draw its graph. 

X — 5x + 6 

4. (a) Find the domain and the range of /(x) = 1 and draw its 

X — 1 X — 2 

graph. 

(b) Compare the graph of part (a) with that of g(x) = 2x . Any conclusions? 

5. Find which of the following functions are one-to one: 

(a) 17x-5 (b) ^JJ+l (c) x'* -x^ (d) x^ (e) x^ +x 

6. Find the inverse, if it exists, of the following functions: 
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(a) y = x^+3,x>0(h)y= — ^— (c) y = x + - (d) y = x^ +- 

3x + 5 X X 

(e) y = 4x^ + x^ , X > 

7. Show that f{x) = x + yjx , x>Q is, one-to-one and find its inverse. 

8. Find the domains of the following functions; determine which of them are 
bounded above and obtain upper bounds whenever they exist. 

(a)x^ (b) 2 + ^— (c) x^-^[xUA (d) 'Vx 
x^+1 

9. Find the domains of the following functions; determine which of them are 
bounded below and obtain lower bounds whenever they exist. 

(a) x'* (b) jcH X (c) Vx - >Ar (d) -jl — 

Vx+1 

10. Determine whether the function is increasing, strictly increasing, decreasing, 
strictly decreasing or not monotone at all. 



1 -+- r^ 

(a) -x^ -X (b) x + — (c) 



^ ■' '^^ ■' ■ ^ '"^ ^^— , x>2 (d) x-x\x>-^ (e) [x] + [xf 



11. Let f(x) = ^[x+— ,x>0 and g(x) = x^ - Ix^ + 12 . Calculate and fmd the 

X 

domains of f + g , f - g , fg , fjg . 
Yl. Calculate the composite function f °g in the following cases: 

(a) / = Vl-x2 'g=^T (b)/=x'--,g=Vl + x2 

1 + x X 

(b) f=h-x' ,g=^x^-2 (d) f=^,g=- 

X X 

13. (a) Let f = x , g = y/x . Calculate / o g . (b) Use (a) to obtain a general rule 
for calculating f ° f'^ ■ 

3.3 Algebraic Functions 

In most applications, a function is defined over an interval or several intervals by a 

3 1 

'formula', for example y = x - x , y = . or y = sin(x) . A particular class 

ylx-1 
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of functions, which are most commonly used in all scientific disciplines are the 
polynomials. 

Definition 3.3.1. The function 

/7(x) = a„x" + a„_[x"~ +... + a[X + 00 , xe 5K (3.3.1) 

where a^ e 31 , < i < n ; a^ jt Q ^ is called a real polynomial of order or degree 
n . The numbers a, , < / < « are the polynomial's coefficients. 

Example 3.3.1. The function y = -3x +5x+ 2 is polynomial of order 2, while 

y = x'' -I is a polynomial of order 7. The constant function >> = 5 is a 
polynomial of order 0. 

The polynomials are a small fraction of a larger class of functions, called 
algebraic functions, which we define next. 

Definition 3.3.2. A function y = f{x) , xe Dt which satisfies 

Pn(x)y" + Pn-i(x)y"'' +... + Pi{x)y + poix) = 0,xeDf (3.3.2) 
where /?,(x) ,0<i<n are polynomials, is called an algebraic function. 

Example 3.3.2. The functions x , r- ,^j\-x are algebraic. Indeed, the first 

\- X 



two satisfy Eq. (3.3.2) with n = \. The function y = ^l\-x satisfies 
y + X - 1 = which has the form of Eq. (3.3.2) with n = 2 . 

Every function that is not algebraic, is called transcendental. A function that 
satisfies Eq. (3.3.2) with m = 1 is called rational. Thus, a rational function is given 

hy y = , where p{x) and q{x) are polynomials. 

q{x) 

In many applications, it is important to find a root r of a given function f{x) , 

i.e., a real number r such that f{r) =0 . A function may not possess any real 

roots. For example, the polynomial x + 1 is always positive and therefore has no 

roots. On the other hand the polynomial x - 3x + 2 has two real roots, 1 and 2. 
If r e 9? is a root of an n -th order polynomial p(x) , then p{x) = (x- r)q(x) , 

where q{x) is an (n - 1) -th polynomial. This result is an immediate consequence 
of the following Lemma. 
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Lemma 3.3.1. For arbitrary a,b e 3i and ne N 

a" -b" =(a-b)(a"-^b + a"-V +... + a^b"-^ +ab"-^) 
The proof using induction, is left for the reader. 

Theorem 3.3.1. An arbitrary polynomial p{x) which has a root re 9? , can be 
rewritten as p(x) = {x — r)q(x) where q(x) is another polynomial. 

Proof. 

n n 

Let p(x) = 2^ai^x .Since p{r) = we have 2,'^k'' =0. Therefore 

k=0 k=0 

n n n 

k^ k^ k=Q 

By virtue of Lemma 3.3.1 we get x -r =(x- r)qi^(x) for some polynomial 
q^. (x) , which implies 

n n 

p(x) = ^(3^ (x* - r* ) = (x- /-)^fl^g^ (x) 

k=0 k=Q 

n 

where q(x) = 2^ai^q^. (x) is a polynomial. This concludes the proof. 

k=0 

If a polynomial p(x) has a root r and can be factorized as 

p{x) = {x- r) q(x) where k >1 and q(r) ^ , we say that r is a multiple root of 
multiplicity k . If r is not a multiple root, we say that r is a simple root. For 
example, p(x) = x -3x +x + l has the simple root 1 and can be factorized as 
p(x) = (x- l)(x - 2x - 1) . The polynomial p(x) = x -3x +4 can be rewritten 
as p{x) = {x — 2) (x + l). Thus, it has one simple root ^j =-1 and one multiple 
root Tj = 2 with multiplicity 2. 

In order to calculate the roots of a function, a priori knowledge of their 
approximate locations is very helpful. Unfortunately, this is not always the case. If 
however, the function is a polynomial, one could use the following result of S. 
Gerschgorin to determine a bound for all the real roots. 
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n 

Theorem 3.3.2. Let p{x) = ^fl^x* , a„ 5^ . If p{r) = , r e 9t then 



r < ' "' ' ' Lii (3.3.3) 

|«„| 

Proof. 

If H^l , Eq. (3.3.3) holds since its right-hand side is always no less than 1. We 

n 

O/^r = , one can rewrite 

k=0 



n-\ 

a„r 



which leads to 



I 1^ r«-i , rn-2 , , ri , ro ^rn-\\ , , ri , Po 

\r\ < -. p + -. jp- + . . . H ■ 1— r- H ■ -— < -. p + . . . + J T + -. — 

r« rn\\'\ \aA\r\ \a„\\r\ "« "« "« 



Therefore 



\r\ < ■ ; T ■ ■ < ■ ■ ; ■ ■ ■ 



which completes the proof. 

Example 3.3.3. Let p{x) = x -3x -16x-12. The absolute values of 
polynomial's roots are bounded by 1+3+16+12=32. Clearly this is not a great 
bound since the actual roots of this polynomial are 6,-1 and -2. Yet, if no 
indication about the roots' locations is available, Gerschgorin's theorem may 
provide useful information. 

Definition 3.3.3. A real number r which is a root of a polynomial 

P{X) = fl„x" + fl„_[X"" + . . . + fljX + flg , fl,j ^ 

where the coefficients are all integers, is called an algebraic number. Any real 
number that is not algebraic is called a transcendental number. 
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Thus, the real numbers which can be divided to rational and irrational numbers, 
can be also divided to algebraic and transcendental numbers. Clearly, every 
rational number is also algebraic. The opposite is not true. For example, the 

number v2 is algebraic ( x -2 = 0) but not rational. The number tt is irrational 
and also transcendental. The proof of these facts is beyond the scope of this book. 

The main reason for the popularity of the polynomials is that they are easily 
manipulated and at the same time may be used to approximate transcendental or 
complex algebraic functions. For example the transcendental function cos(x) can 

be approximated by the quadratic polynomial p(x) = 1- x /l for 'small' X . How 
small? Once the reader controls the concepts of differentiation and infinite series, 
he should be able to show that by approximating cos(x) by p{x) we obtain a 

'relative error' of less 0.3%, provided that Ixl < 0.5 , namely, 



p(x) - cos(x) 



cos(x) 



<310 



-3 



X < 0.5 



The quadratic p{x) is plotted vs. cos(x) in Fig. 3.3.1. over the interval [-l,lj. 

The maximum deviation is at the endpoints, but even there it does not exceed 
over 7%. 




Fig. 3.3.1. Approximating cos(x) (solid) by p(x) =l-0.5x (dashed). 



3.3 Algebraic Functions 
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The most commonly used transcendental functions are the trigonometric 
functions such as sin(x),cos(x),tan(x),cot(x) , the exponential functions such as 

10^, e^ (the number e will be defined later in this chapter) and their inverses. 

Other functions that are not algebraic, are functions such as [xj , Ixl and 



fix): 



, X rational 
1 , X irrational 



(3.3.4) 



Definition 3.3.4. A function f(x) , defined over the real axis, is called periodic, if 
there exists a real number T such that 



f(x + T) = f(x),xe3^ 



(3.3.5) 



The number T is a. period of f(x) . 

It is easily seen that if T is a period of f(x) , so is -T . 

Example 3.3.4. f{x) = sm(x) is periodic (Fig. 3.3.2) with a period 2;r, i.e. 
sin(x + 2;r) = sin(x) for all x . Clearly, 4;r is another period of sin(x) and so is 
any integer multiple of l/r . However, the number l/r is the smallest of all the 
periods of sin(x) . 



Fig. 3.3.2. The periodic function sin(x) . 



Polynomials, unlike trigonometric functions, are never periodic. 

Definition 3.3.5. Consider a function /(x) , - a < x < a . If /(x) = /(-x) for all 
X^ Df, the function is called an even function and if /(x) = -/(-x) for all 
xe Df , the function is called an odd function. 



b 


, XiXj = 


c 


a 




a 
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Example 3.3.5. The functions x ,cos(x),|x| are even functions, while x ,tan(x) 
are odd functions. The functions 2 * and x - x are neither even nor odd 

functions. The function yjx is not defined for x < and is therefore neither an 
even nor an odd function. The function defined by Eq. (3.3.4) is even since a real 
number x is rational if and only if -x is rational. 

PROBLEMS 

1 . Let Xj , X2 be the two roots of the quadratic polynomial ax +bx + c . Show 



X, +X', 



2. Show that f{x) = ^x+x is an algebraic function. 

3. Show that /(x) = -Jx + yx is an algebraic function. 

4. Use induction to prove Lemma 3.3.1. 

5. Find a bound to the roots (absolute value) of the polynomial lOOx + x - 1 . 

6. Find a bound to the roots of the polynomial X -x-100. 

7. Show that the following are algebraic numbers: 

(a) V2+V3 (b) V2+V2 (c) V2V3 

8. Let a , an algebraic number, be a root of a quadratic polynomial. Show that 
the number a + r is algebraic for arbitrary rational number r . 

9. Let a , an algebraic number, be a root of a quadratic polynomial. Show that 

a is algebraic as well. 



10. Let a , an algebraic number, be a root of a quadratic polynomial. Show that 

a " is algebraic for all n > 1 . 

1 1 . Find the smallest period of the following functions: 

(a) cos(3x + 1) (b) tan(x/2) (c) sin(x) + tan(x) 

12. Find a domain where the function y = x +x +^jx is even. 

13. Show that the sum and difference of two even (odd) functions is also even 
(odd) functions. 

14. Show that the product of two even (odd) functions is also an even function. 

15. Let /(x) , g{x) be even and odd functions respectively. What can you say 

about the functions: (a) f{x) + g{x) (b) /(x)-g(x) (c) /(x)-g(x). 
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3.4 Sequences 

In this section we introduce the basic notations that distinguish calculus from other 
subjects in mathematics: sequence, limit and convergence. 

Definition 3.4.1. The range of a function f '■ Dc -^ 5R , whose domain is an 
infinite subset of N - the set of natural numbers, is called a sequence. 

If the domain of / is all of N , the sequence is composed of /(I),/ (2),... which 
are the terms or elements of the sequence. The general term is denoted by 
/(M)and the sequence is written as {/(I), /(2),. ..,/(«),...} or, using subscripts, 

as {/[, /2, ...,/„,...} . Representing a sequence using subscripts emphasizes the 
existence of order among the sequence's terms: /; - first term, /j - second 
term, . . ., /„ - « -th term and so on. If we change the order of the terms we get a 
different sequence. We usually assume one of two cases: (a) Df=N or (b) 
Df = N -{\,2,...,m-\} for some integer m>\ . This implies that the subscripts 
are consecutive positive integers. A sequence {ai,a2,---,fl„,---} is denoted by 
{a„}~^i or simply by {«„}. A sequence {fl„,,a„+i, ...,«„,...} is denoted by 



Example 3.4.1. The function a{n) = n + 1 , n > 1 defines the sequence 
{2,5,10,17,. ..,« +1,...} also written as {n +l}JJLi or as {n + 1} . The function 

a{n) = ,n>3 detmes the sequence <6, —,—,..., ,...> which we can 

n-2 [ 2 3 n-2 ] 

write as < > . The function a(n) = l + (-Y)" ,n>l defines the sequence 

1«-2J„.3 

{0,2,0,2,...,l + (-l)",...}. 

Consider now the sequence {a„}~^3=< > . If we calculate 'enough' 

1«-2J„.3 
elements, we see that a^ 'approaches' 1 as the n increases. In fact, the difference 

between a^ and 1 is 1 = . In the previous chapter, we showed, that 

n—2 n—2 

for any given number e>0 , we can find a natural number n^ie) (i.e. «q 

depends on f ) such that — <£ for all n > MqC^) • Similarly we can show (but 

n 

leave it for the reader), that for any given number e >0 , we can find a natural 
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number Mq (^) such that < e for all n> Hq (e) . In other words, our 

n-2 



sequence's elements a„ 



n + 3 



get as close as we want to the number 1. Thus, 



n-2 
by letting n increase indefinitely, the elements of the sequence 'converge' to 1. 

We now present a rigorous definition of the 'convergence' notation for 
sequences. 



Definition 3.4.2. A sequence {«„} converges to A as m— >c>o ^ if for every 
£ >0 , there exists a positive integer Mq (^) such that 



n > «o(^) ^ P« ~ ^ < ^ 



(3.4.1) 



The number A is called the limit of {«„} . A sequence that converges is called a 
convergent sequence. Otherwise, the sequence diverges and is called a divergent 
sequence. 



n^-1 
Example 3.4.2. Let a„ = — ^ 

2m + m + 1 

denominator by « , we obtain 



. By dividing both the numerator and the 



. 1 1 

2 + - + ^r 



n n 



The numerator and the denominator 'seem' to approach 1 and 2 respectively as n 
increases. Consequently, we are motivated to believe that A = — is limit of the 
sequence. In fact, we easily get 





1 


3 






2 




n 


n 




1 


1 ^ 


2 


2 + - 


+— 




n 


n J 



1 3 
n n 



1 4 

— + — 



^1 ^ 1 1 

2 2 + - + — 



n n 



An 



Now, given an arbitrary f > we can guarantee 



— <e or n> — . This suggests the choice Ma (^ ) = 
An Ae 



Ae 



< s , provided that 
+ 1 and the sequence 



indeed converges to 



1 
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If a sequence {fl„ } converges to A , we denote it by lim a^ = A . For example, 
in the previous example we proved 



lim- 



n^-\ 1 



■2n+n + l 2 



The convergence notation can be illustrated geometrically as follows. Consider the 
X- y plane where the natural numbers are marked along the x - axis and the 

sequence's elements - along the y- axis (Fig. 3.4.1). The sequence {a^} 
converges to the number A , if and only if, for every e >0 , except for a finite 
number of elements, all «„ are inside the strip A- £ < y < A + e . The number of 
the 'outsiders' may increase when e decreases, but it is always finite. The 
sequence that in fact produces Fig. 3.4.1, is a,^ =[n + sm(2n)]/[2n + cos(n)] 
which converges to 1/2 . The illustrated case is £■ = 0.05 for which only 7 
elements are out of the strip. 



A+e 



As 



+ 


+ 


, ++ 


+ 


+ 




+ 
+ + + 


+ 

1 + 


, ^+ ,+ ++ + -^ 




+ + 


+ 






+ 


' ^ .-^ 


' + 

+ 


'+ + + ++ + ++ 



Fig. 3.4.1. Illustrating 'convergence'. 

The next result follows directly from Definition 3.4.2 and its proof is left as an 
exercise for the reader. 

Theorem 3.4.1. A sequence {a„ } converges to A , if and only if, the sequence 
{ii„ } = {fl„ -A] converges to 0, i.e. lim a„ = A <=> lim (a,, - A) = . 
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The question whether a limit of a sequence is unique, is answered clearly by the 
next theorem. 

Theorem 3.4.2. Consider a sequence [a^ } . If lim a^ = A and lim a„ = B then 
A = B. 

Proof. 

Assume Ait B . For any given e >0 there are «[ (e) , M2 (^) such 
that |fl„ - AI < f , « > «] (£■) and |a„ - bI <£■,«> M2 (^ ) ■ Therefore, 

|fl„ - AI < f , |fl„ -BI < £■ for M > «o = max [«[(£■), M2(^)] and consequently, by 
using the triangle's inequality 

I A - B| = I A - fl„ + a„ - B| < I A - a„ I + |fl„ -B\<2£ , n> n^ 

\a-b\ 

The particular choice e = leads to contradiction, i.e. A = B . 

Example 3.4.3. Consider the sequence a„ = ^1 + i/n - 1 , m > 1 . If we multiply 
and divide the general term by yjl + 1/n + 1 , we obtain 



1 
n 



1 + 1+1 



n 



and consequently limfl„=0. By virtue of Theorem 3.4.1, we also have 
lim ^Jl + l/n = 1 . 

Definition 3.4.3. A sequence {a„} is called bounded, if \aJ< M ,ne N for 
some M > . 

The sequence {a^ } = {«} is not bounded, while {a„ } = < > is bounded by 2. 

I n J 

A bounded sequence may not have a limit. For example, the sequence {1,0,1,0,. . .} 

does not converge. On the other hand, the opposite is true. 

Theorem 3.4.3. A sequence which has a limit, must be bounded. 
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Proof. 

Let lim a„ = A . Then, for f = 1 , an iiq (1) can be found such that |fl„ - AI < 1 for 
all « > MqCI) ■ Therefore, la^l < 1 + Ul , « > «o(l) • The choice 

M =max[|a, Ufl,!,..., fl„ ni ,1 + |A|] 
I ^11 ^r I "o'-^-'i I I 

guarantees |a„ I < M ,ne N . 

Example 3.4.4. Consider the sequence 

«^l<«<lo*' 

" n>lO' 



2m + 1 

The sequence converges to 0.5 and is therefore bounded. It does consist of some 
large elements such as aioooo=10 but is still bounded, for example, by 
M=10'^ 

Example 3.4.5. The sequence {^Jn } is not bounded. Indeed, if this were false, one 

could find a number M > such that y/n < M for every natural number ne N . 

However, this last inequality does not hold for n = (\m\ + 1) , i.e. the sequence is 
not bounded. 

Definition 3.4.4. A sequence [a^ } is called bounded away from zero, if there 
exist a number p>0 and a positive integer m , such that |a„ I > p for all n> m . 



3 5 9 
Example 3.4.6. The sequence < 1, 0, 0, — , — , — ,...> is bounded away from 0, since 

|a„ I > — , « > 3 . The sequence l — > is not bounded away from and so is the 

2 [n] 

sequence <1,— ,1, — ,..., 1, —,...> (why?). 
2 3 n \ 



A sufficient condition for a sequence to be bounded away from 0, is given next. 
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Theorem 3.4.4. A sequence {a^^} that converges to A^^O, is bounded away 
from 0. 

Proof. 

\A\ 
Let e = ^—^ . Since {a^} converges, we can find nQ(e)>0 such that 

a„-A<-i — - for all «>«{,(£•). The triangle's inequality implies 

Ul Ul 

II I I 

|A| - fl„ < |a„ - A| < ■*— *■ for n>nQ{e). Consequently a„ >■'—'-> ,«> MqC^) > 

i.e. {fl„} is bounded away from 0. 

The next results are very useful in determining whether a given sequence 
converges. 

Theorem 3.4.5. Consider the sequences {«„ } and [b^ } where < a„ <b^ , m > 1 . 
If lim b^=0 , then lim a„ = . 

Theorem 3.4.6. If a sequence {a^J converges to A , then [ca^^] converges to cA 
for all c e 9? . 

Theorem 3.4.7. If the {|a„|} converges to 0, then {a^} converges to as well. 

The proofs of these theorems are left as an exercise for the reader. 

Example 3.4.7. Let a^ =q" ,n>l where < ^ < 1 . Then; lim «„ = 0. 

Indeed, rewrite q = where p = 1>0. Using Bernoulli's inequality 

1+/7 q 

we get 

(l + pT>l + np>0 

Since (1 + p)" > and l + np >0 we also have 

(1 + p)" 1 + np np 
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The sequence < — > converges to 0. Thus, by virtue of Theorems 3.4.5-6, < — 

[n] [np 

and {a„}=^ [ all converge to as well. By implementing 



l + wpj [(l + p)" 

Theorem 3.4.7, we get lim q" = for bl < 1 . 



One of the most important features of the convergence notation is the following 
result. 

Theorem 3.4.8. If the sequences {a,,} and {b^} satisfy a^=b^ ,n> jiq for some 
positive integer Mq and lim a„= A, then lim b^= A as well. 

We thus conclude, that the head of a sequence, defined as the first Mq elements 
where Hq is any fixed positive integer, plays no role in deciding whether or not 
the sequence converges. The convergence or divergence of the sequence is 
determined solely by remaining elements. 

The proof of Theorem 3.4.8 is quite trivial and is left for the reader. 



PROBLEMS 

1. Find which of the following sequences converge and obtain their limits. 

(a) (1,-2,3,-4,5,-6,...) (b)|--} (0 {—,—, — , 
(d){l,-l,^,-l,^,-l,i^,...l (e)l^ ^^\nH-innm 
n + M — 1 



2 4 8 1 \ n \ I n 

r 2 

I n 
(g) 



2m^+2«-5 



M-3 

2. Let a„ = . Find n,-. such that 

" 7« + 2 *' 



< 10 ,n>nQ. Does finding n^ 



guarantee that {«„} converges to — ? Converges to any limit? 



3. Let a = j= — . Find Ma such that 

1-3Vm 



1 

a„ +- 

" 3 



_'3 

< 10 , n> Hq . 
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4. Let a„ =1 + q + ... + q" ,\q\<l . Find lim «„ if it exists. 

5. Show that the following sequences converge to 0: 

6. Let lim a„ = A and define /?„ = — ^^ ^^ ,neN. Prove: lim /?„ = A . 

7. Let a„ =l-\ 1 1-... + — . Show that {a„ } is unbounded. 

2 3 n 

8. Let a„ =l-\ 1 I-...H — . Show that {a„ } is bounded. 

2! 3! m! 

9. Use the binomial theorem and problem 8 to show that the sequence 

iVl 

1 H — > is bounded. 

nj J 

10. Leta„=l + + + ... + , M>1. Show that lima,, =2. 

1-2 2-3 {n-\)-n n^~ 

1 4 3n - 2 

1 1 . Let a„ = 1 h . . . H . Use the result of problem 7 to show 

2-3 5-6 (3«-l)-(3«) 

that {a^ } is unbounded. 

12. Let {a„},{Z7„} denote convergent and divergent sequences respectively. 
Show an example where; (a){c„} , c^ =a^Z7„ is convergent (b) 
{c„ } , c„ = a,J}„ is divergent. 

13. Let {a^ } , {^„ } denote convergent and divergent sequences respectively. What 
can you expect from the sequence {c^} , c^ = a^+b^l 



3.5 Basic Limit Theorems 

The following theorems significantly extend our ability to determine whether a 
given sequence has a limit. We start with a limit theorem related to basic 
arithmetic operations between sequences. 

Theorem 3.5.1. Let lim «„ = A and lim b„=B . Then 

(a) lim(fl„ +b,J = A + B 

(b) lim(a„-fe„) = A-B 

(c) lim(a„Z7„) = AB 
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(d) lim 



kKj 



— , provided that Bi^Q and b„ ^0 ,ne N . 



Theorem 3.5.1 yields that the sum, difference, product and quotient sequences, 
of two arbitrary convergent sequences, converge as well. The only restriction is 
in part (d), where we must assume that neither the elements of the sequence {b^ } 
nor its limit may vanish. However, if B ^ there exists some me N such 
that Z?,, ^ , M > m (why?). Consequently, we may replace part (d) with 

(a y A 
lim — ^ = — , where B ^0 and b„ ^0 , n>m . 
B 



Proof. 

g 

(a) Given an arbitrary f > , let us define e^ = — . Since {a„ } converges to A , 

we can find Mj (e^ ) (denoted by Mj ) such that |a„ - AI < fj ,«>«[. Similarly, 
we can find Mj (^i ) (denoted by Mj ) such that \b^^ - B\<£^ , m > Mj ■ 
Consequently 

\{a„ +b„)-iA + B)\ = \ia„-A) + ib„ - B)\<\a„- A\ + \b„- B\<2ei =e 

provided that « > Mq = max(M[,M2) • By virtue of Definition 3.4.2 part (a) is 
valid. 

(b) The proof is almost identical to that of part (a) and is left for the reader. 

(c) In order to show lim a^b„ = AB we rewrite 

a„b„ -AB = ia„b„ - Ab„ ) + (Ab,, - AB) 
and get 

\a„b„ - AB\ < \a„b„ - Ab,\ + \Ab„ - AB\ = \b^\a„ - A\ + \A^„ - B\ 

Since a convergent sequence is also bounded, we can find M > such 
that |fe„ I < M , « e N . Consequently 



\aj}^ - AB\ <M\a^-A\ + \A\b^ - B\ 
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Consider an arbitrary e >0 and let s^ 



and Eo 



Since lima„ 



2M 2\A\ 

we can find Wj (e^ ) such that |a„ - AI < £■[ is valid for n> n^. Similarly, since 
lim b^= B we can find «2 (^2 ) ^^ch that \b^ - Z?| < £"2 is valid for m > M2 . 

n— >oo 

Therefore, by taking Mq = max(«i , «2 ) , we obtain 



\a„b„- AB\<Me, +\A\£t =M + \A\—r-r = £ , n>nn 

\ n n I 1 I I 2 2M I I2A 



and the proof is completed, 
(d) Rewrite 



^n A_ a„B-b„A _ a„B-AB + AB-b„A _ a„-A ^ ^(B b ) 
b„ B b„B b„B b„ b„B 



^n ^n 



which implies 



a^_A 
b„ B 



\a„ - A A , 

^ 11 +^\^-K 
k \b..By 



Since {i>„ } converges to B i^Q and since none of the elements vanishes, the 
sequence {b^ ] is bounded away from zero, i.e. we can find M > such that 
|Z7„| > M for all n . 
Consequently 



a^_A 
b„ B 



M 



■A\ A , 



M\By 



Me M\B\e 

Now, consider an arbitrary e >0 . Define e, = and f, = — 'r-\— . Choose 

2 ^ 2|A| 

«i(fi) and ^2(^2) &^ch that la,, - AI < fj ,n>n^ and |i>„ - bI < ^2 , m > M2 ■ This 
is feasible since {a^ } converges to A and {Zj^^ } converges to B . Thus, 



a^_A 
b„ B 



M 



M\B\ 



\B-b\<- 



\A\So 



£ e 

- + — = £ 



M M\B\ 2 2 



for n> Hq = max(Mi,M2) and this completes the proof of part (d). 
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It should be noted that the particular choices of fj , S2 in the proofs of parts (c) 

and (d), were neither lucky guesses nor a demonstration of special expertise. It is 
merely a simple, standard procedure, which adds a touch of elegance to the proof. 
For example, consider the proof of part (c). After obtaining the inequality 

\aj3^ - AB\ < M\a,^ ~ ^| + 1^||^« -B\,ne N 

we continue the proof differently as follows. For a given f > we have to show 
the existence of Mq (e) such that |a„^„ - AB\ < e ,n> Uq. However, for an 

arbitrary e > (it is irrelevant whether we use the symbol e or £■ ), we can find 
«i (£■ ) such that |fl„ - AI < £■ is valid for n> n^. Similarly, we can find «2 (^ ) 

such that |Z7„ -£!<£■' is valid for « > W2 • Therefore, by taking «q = max(Mi,M2) , 
we obtain 

|fl„Z7„ - AB|<Mf + |A|£-* =(M£-* +|A|£-*) = (M +|A|)£-* , n>nQ 

Since this is true for arbitrary £■' > , it is true also for the particular choice 

* s 

In other words, there exists n^ such that 



M +\A\ 

\a„b„ -AB\<(M +\A\)£ =(M+\A\) r^ = £, n > n,. 

\ n n I V I |/ I K^^_^|^|) 

and the proof is concluded. Thus, instead of having to pre-calculate 'clever' 

I I * 

choices forfj and £"2 , we obtain an inequality such as m„ - C/ < cf ,n> Hq and 

apply it for f = — . 
c 

Theorem 3.5.2. Let a^,>0,neN and lim a„ = A . Then lim 'ifa^ = '^A for 
arbitrary me N . 

Proof. 

Clearly A > (why?). Define b^ = "\[^ ,ne N and B = ^ . We distinguish 
between two cases. 

(a) A > which implies B > . Rewrite 

a„-A = Z?;" -B"'= (b„ - B)(b'^-' + b;:'-^B + ... + b„B"'-^ + B""-') . 
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Since B>0 and b^>0 ,ne N we obtain 



= /7„-Bp„ +&„ B + ... + b^B +B \>B \b„-B\ 



Therefore \b„-B\<- -r- which implies \m\b„=B (why?). 

(b) A = which implies Z? = . We have to show limZ7„=0, i.e., that for 

arbitrary e >Q there exists n^e N such that |^„ I < ^ for n> Hq. Define fj = f " . 
Since lim a„ = , one can find HqE N such that |fl„ I < f'j ,n>nQ. Thus 

'V '^n < ^^1 = ^ for n > "o ■ This concludes the proof. 

Example 3.5.1. Let a^ = yjn +2n -yjn + 1 , n > 1 . In order to find whether the 
sequence converges, rewrite 

(V«^+2m - V«^+1)(V«^+2m + V«^+l) 2n - 1 



■^r? +2n+^n- +1 Vn^+2n+V«^+l 

which, after dividing the numerator and the denominator by m , leads to 



2-1 



n 



By Theorem 3.5.1 part (b), the numerator converges to 2. Indeed, it is the 

difference sequence of {2,2,2,...} and l — \ that converges to 2-0 = 2. The 

[n] 

i ( if 
denominator converges to 2 as well. First, lim — r- = lim — = . Therefore 



lim 1h — 7 =1 leading, by Theorem 3.5.2 to lim llH — 7=!- Similarly, 



lim J^ + — =1 and the whole denominator converges to 2. Therefore, lim a„ = 1 . 

n «->~ 

The following results are common knowledge to the reader who is familiar with 
elementary trigonometry: 
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(Rl) limsin[-l = (R2) limMsin|-l = l (R3) limcos|-l = l 

Result (R2) follows from the inequality sin(x) < x , < x < 7r/2 (proved in 
Section 5.3) while (Rl) and (R3) follow straightforward from (R2). 

T7 1 -ICT T * (« + 3)(Vm + V« - VM ) rr f , -f ,. 

Example 3.5.2. Let a„ = . To iind ii lim a„ exists we 

(n - 1)(« - 2) sin(l/M) «^~ 

. (n + 3) (V" + yln - 4n) rp. . ^. « + 3 

first rewrite a,j = '^ . The traction , rewritten as 

(n - 1) {n — 2) sin(l/M) n — \ 

1 + 3/m 



l-l/« 



, converges to 1 (Theorem 3.5.1). The next expression is 



/ ~r r~ (\n + \ln -'Jn)(Jn + 'Jn +^Jn) -Jn 

■^n + ^jn -yjn =—^ . = . = 



^Jn + ^Jn + -Jn •\ln + -Jn + -Jn ■\jl + l/v« + 1 



which by virtue of Theorems 3.5.1 and 3.5.2, converges to 0.5. The last expression 
(M-2)sin(l/«) = Msin(l/M) -2sin(l/«) , converges (results Rl, R2 and Theorem 

3.5.1) to 1. Consequently, lim a„ = 0.5. 

Example 3.5.3. Let a^ =yla , ne N for a fixed a > 1 . Clearly fl„ > 1 for all 
ne N , i.e. a„ = l + «,j , a„ > . Therefore, by Bernoulli's inequality: 

a = (1 + «„)" >l + na„ and consequently <a„ < . Using previous result 

n 

we obtain lim «„ = and finally lim a„ = 1 . If < a < 1 we have 

lim nj— = —j= = 1 (why?) and therefore lim 'ija =\ as well. Thus, lim 'ija =\ for 
arbitrary a > . 
Example 3.5.4. Consider the sequence 

n -Jn + -\Jn — -sjn . ( 1 
sinl 



cos(l/n)^„ + ^_^ ^3/^ 

Multiply and divide the midterm's numerator by -\/m + V« +v« and its 
denominator by ^m + V« + V« . The midterm can be therefore rewritten as 
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-\/n + v« - v« _ Vn (■\ln + yjn + V") 

^jn + ^Jn—^Jn -Jn (-^Jn + yn + -Jn) 
which implies 

Vm sin(l/VM) a/« + Vm + Vm a/m sin(l/V«) vl + 1/^/« + 1 

{I = ' = ' 

^jcos(l/n) ^n + ^4^+4^ ^cos(l/n) ^i^ij^f^ ^i 
A multiple use of Theorems 3.5.1 and 3.5.2 shows lim a„ = 1 . The details are left 

n— >oo 

as an exercise for the reader. 

Theorem 3.5.3. If the sequences {«„ } ,{b^} , {c„ } satisfy lim «„ = lim b^ = A 

and a,j <c^<b^,neN then lim c„ = A as well. 

n— >oo 

Proof. 

For arbitrary ne N we have 

|c„ -A\ = \c„ -a„+a„-b„+b„-A\<\c„-a„\ + \a„ -b„\ + \b^-A\< 2\a„ -b„\ + |Z)„ - A\ 

The right-hand side converges to which implies lim|c„ - AI = and concludes 

n— >~ 

the proof. 

Example 3.5.5. Let a„ =^,b„ =1 + — >c„ =1 + — r-. Here lim «„ = lim fe„ = 1 

n n "^°° «^°° 

and 1 < 1 + — - < 1 + — . Therefore, lim 1 + — - = 1 . 

n n n^~V n J 

PROBLEMS 

1. Find the limits of the following sequences: 

1 ^fn ^, ^ sin(M) ^ ^ ^2 N • 2^2 

a. a„= — + -j= (b) a„ = — ^^ (c)a^=(n +M)sin — 



(d) fl„ = = tan 



^ 2 



2vM +5 I M + vm 
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2. Let lim a„ = A, lim £>„ = B , lim c„ = C . Prove lim (a^jb^c^ ) = ABC . 

3. Extend the result of problem 2 to m convergent sequences for arbitrary 
me N. 

4. If a sequence {a„ } does not converge to A , there must some £ = £o such that 
for any positive integer m , there is another positive integer mQ> m for which 

a,„ - A > fg . Use this argument and the binomial theorem to show 
Um Vn = 1 . 

5. Use Example 3.5.3 to calculate lim v3""'" . 

6. Let < a < y5 . Define a„ = yja" + /3" ,ne N and prove lim fl„ = /? . Does 
this also hold when a = pi 

a" 

7. Prove lim — = for arbitrary a e 9? . 

«->»= n\ 

fn 

8. Let lim a„ = A where a„ >0 , ne N . Show that for an arbitrary fraction — 

m 



such that m, A: > 0, the sequence {i/fl,, } converges to A'^ . 
9. Let lima,, = A . Define b„ = ""+""+'+ '^"+2 and find limZ7„ if it exists. 



10. Let lima„ =A, limZ?,, =B. When does lim J '^^ exists? If it 

exists, calculate it. 

1 1 . Let «! = ^2 = 1 and for n> 2 define a,, = ^^ and assume lim a,, = A . 

««-2 "^~ 

Find A . 

12. Let {a„ },{£>,,} denote two divergent sequences. Is the sequence {a,, +fo„} 

divergent? 

13. Show that for arbitrary rational number r there is a sequence of irrational 
numbers that converges to r . 

14. Let {a,j } ,{b„} denote two divergent sequences. Is the sequence {a„^„ } 
divergent? 

15. The sequence Gj = 1 ; a„+i = — ,n>\ is a typical continued fraction 

1 + ^— 

l + a„ 

(the name refers to the structure of a„+i , once we express a„ in terms of a„_i 
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etc.). Find lim a„ , if it exists, then calculate the first 10 terms of the sequence 

and compare. 
16. Show that the sequence in problem 15 is decreasing and apply Theorem 3.7.1 
(Section 3.7 below) to show that the sequence converges. 



3.6 Limit Points 

Definition 3.6.1. Let S denote an arbitrary set of real numbers. A number se3l 
will be called a limit point (also accumulation point or cluster point), if for every 
£■ > there exists a number r e 9? such that \r — s\< e , r i^ s . 

The requirement \r — s\< e , r i^ s can be replaced by the requirement 
< Ir - il < ff since < Ir - il <=^ r ^^t 5 . An immediate consequence of Definition 
3.6.1 is the following theorem. 

Theorem 3.6.1. Let se'H be a limit point of S . Then S is infinite, and there 
exists a sequence {r„ } such that: 

(a) r^e S ,ne N 

(b) A: ^ / =^ r^ ^ /-; 

(c) lim r„ = s 

Proof. 

Since j' is a limit point of S we can find r^e S such that Ir; - i'l < 1 , rj ^ .v . 
Define e^ = . Clearly e^ >0 and therefore there exists /■2 e 5 such that 



72 - 5 < f 1 , r2 ^ j' . Since /■2 - ^ < , we must have /-j ^^ r2 . In general we 

|r„ -s\ 11 

choose Ejj = > and find r„^.i e S such that r„^.i -5 < f„ , /-^^.i ^ s . We 

thus constructed an infinite sequence which satisfies (a) and (b) (therefore S is 

infinite). By using induction we get \r ., -s\< ,neN and (c) is concluded 

2" 
as well. 

Example 3.6.1. The set ^i = {1,2,. ..,17} is finite and therefore has no limit points. 
The set ^2 =0, — ,2,— ,3, — ,4,...> has a single limit point s = 0. A sequence. 
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which is a subset of ^2 , whose elements are mutually distinct and which 

. n • f 1 Jl 1 1 It,, . c Jl 3 1 7 1 15 

converses to 0, IS tor example <—, — ,—,... ^ . The set 5, =<—,—, — ,—,—, — , 

[2 3 4 J ^ [2 4 3 8 4 16 

has two limit points, namely and 1. 

By definition, the elements of a set are mutually distinct. This is necessarily true 
for a sequence. Therefore the concept of 'limit point' for a sequence is defined 
differently. We first introduce the 'subsequence' notation. 

Definition 3.6.2. A sequence {Z?,, } is called a subsequence of a sequence {a^ } if 
bi^=a„ ,keN ; Mj < «2 <•••<"*< ''/t+i < ••• (3.6.1) 

Definition 3.6.3. A number 5 e 9? is called a limit point (also accumulation point 
or cluster point) of a sequence {a,,} if there exists a subsequence {«„ } of {«„} 

such that lim a„ = s . 

Example 3.6.2. The sequence {a„} = {0,1.9,0,1.99,0,1.999,...} has two limit 
points. The first is which is the limit of the convergent subsequence {0,0,...} . 
The second is 2 which is the limit of the subsequence {1.9,1.99,1.999,...} . 

Theorem 3.6.2. \i s is a limit point of a set S , bounded above by M , then 
s<M . 

Proof. 

\f s> M then s = M +a ,a>Q . Since 5 is a limit point of 5 , we should be 

able to find r e 5 which satisfies < r - .s < — . Therefore, 

' ' 2 

cc 
s <r-\ — <r + a<M +a which contradicts s =M +a . 

2 

Corollary 3.6.1. The set K of the limit points of a bounded above set S , is also 
bounded (by the same bound) and therefore possesses a supremum. 

Definition 3.6.4. If the set K of the limit points of a bounded above set S is 
nonempty, its supremum M is called the limit superior of S and is denoted by 

M = limsup^ = iimS (3.6.2) 
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Similarly, the set K of the limit points of a bounded below set, is also bounded 
below (by the same bound) and therefore possesses an infimum. 

Definition 3.6.5. If the set K of the limit points of a bounded above set S is 
nonempty, its infimum L is called the limit inferior of S and is denoted by 

L = liminf 5 = limS (3.6.3) 

Theorem 3.6.2 and Definitions 3.6.4 and 3.6.5 can be extended for sequences. 
This is left as an exercise for the reader. The next result however, does not hold 
for sequences. 

Theorem 3.6.3. Let s he a limit point of a set S . Then, for every e>Q the 
interval {s- e,s + e) contains an infinite number of elements from S . 

The proof is straightforward and is left for the reader. 

Example 3.6.3. Consider the set 5i ={-5, (-4,-1), 0,2, [3, 5], 7, 8, 1000} which 
consists of six discrete integers, the open interval from -4 to -1 and the closed 
interval from 3 to 5. The set is bounded and its infimum and supremum are -5 
and 1000 respectively. Each number within the two intervals is a limit point and 

limSi = -4 , iimS'i = 5 

Now, consider the sequence ^2 = {-2,3,1,-1,1,-1,1,-1,...} . Here the infimum 
and the supremum are -2 and 3 respectively. There are only two limit points and 

\m^^ = -1 , limS2 = 1 

The following results can be easily derived from Definitions 3.6.4-3.6.5 and their 
proofs are left for the reader. 

Theorem 3.6.4. Let S be an infinite set and M a real number. Then, M = limS if 
and only if, for every e>Q there is only a finite number of elements in S that are 
greater than M + e while there is an infinite number of elements in S that are 
greater than M -e . 

Theorem 3.6.5. Let S be an infinite set and M a real number. Then, M = limS if 
and only if, for every e>Q only there is a finite number of elements in S that are 
less than M - e while there is an infinite number of elements in S that are less 
than M +e . 
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The next result presents an important relation between a sequence and the class of 
its subsequences. 

Theorem 3.6.6. A sequence [a^ } converges to A , if and only if all its 
subsequences converge to A . 

Proof. 

Let A = lim a„ and let {bi^ } = {a„ } denote an arbitrary subsequence of {a,,} . For 

a given f > we can find Uq such that \A - aJ < e , n > Hq . Using Eq. (3.6.1) we 
easily obtain rii^ >k for all k . In particular, «„ > Hq and consequently, 
|A-Z7„|<f for n>nQ. Thus, every subsequence of {a„} is convergent and 
converges to A . Now, assume that every subsequence of {a^ } converges to A . 
Since {a,,} is also a subsequence of itself we have A= lima„ and the proof is 

completed. 

PROBLEMS 

1 . Find limit points of the set 

■li,l-l,-ii,2,-2,-l,i,3,-3, 
2 3 4 5 6 

Observe S as a sequence and determine whether this sequence is convergent. 

2. Find all the limit points of the sequence {1,2,5,1,2,5,1,2,5,...} . Does the set 
which consists of all the mutually distinct elements of the sequence have a 
limit point? 

3. Find the supremum, infimum, limit superior and limit inferior of the sets: 

(a) 5i ={0,1,0,2,0,3,...} 
(b)52 ={4,3,2,...} 

'2 ,4 ,4 fi 



(c)53=^-,-l,-,-l,-,-l,-,... 

^[3355 
(d)54={-5,[-4,0.1],6,7,10,(-8,-6)} 

4. Find the supremum, infimum, limit superior and limit inferior of the 
sequences: 
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(a) 5i ={0,1,0,2,0,3,...} 



(b)52 



(d)54 



-1, 



l,-,-l,- 



i' '3' '5 
(c)53 ={1,5,1,5,1,5,...} 

2' 2'^' 4'9 



8 '27' 



5. Let {a„ } ,{b„} be bounded above sequences. Show 

Iim(fl„ +b„)< lim(a„ ) + lim(Z7„ ) 
if both sides exist. 

6. Let {a^ } , [b^^ } be bounded below sequences. Show 

lim(a„ +b„)> lim(a„ ) + lim(Z7„ ) 
if both sides exist. 

7. Let {fl„},{Z7„} be bounded above sequences such that a^<b^ ,n> jiq for 

some fixed «o • Show linw^ < limZ7„ if both sides exist. 

8. Make a necessary change in Theorems 3.6.4-3.6.5 and extend them for 
sequences. 

9. Consider the sequence <1,0,— ,0,— ,0,...L Find two subsequences that 

converge to the same limit. Is this sufficient to guarantee the convergence of 
the sequence? 

10. Let 5 = {1.1, 10 ^1.01, 1.001, 10 ^1.0001, 1.00001, 1.000001, 10 ^...}. Does 
the sequence converge? If not, find sup 5 , lim^ , inf 5 , Jims' . 

11. Let 5 ={o.9, 72,0.99,^3,0.999,^4,...}. Find sup5 , iim5 , inf 5 , lim5 . 

You may need to use the result of problem 3.5 and Example 3.7.2 (Section 3.7 
below). 



3.7 Special Sequences 

In this section we introduce and investigate special types of sequences, such as 
monotone sequences, sequences that converge to infinity and Cauchy sequences. 
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3.7.1 Monotone Sequences 

Definition 3.7.1. A sequence {a^} is called monotone increasing, if a^^<a^^^ for 
all « . If fl„< a„+i for all m , it is called strictly monotone increasing. 

Definition 3.7.2. A sequence {a^^] is called monotone decreasing, if a i^>ai^^i for 
all n . If fl„> fl„^.[ for all m , it is called strictly monotone decreasing. 

Clearly, every strictly monotone increasing (decreasing) sequence, is also 
monotone increasing (decreasing). 

Example 3.7.1. The sequences {1,1,2,2,3,3,...}, {1,1,1,2,3,4,5,...} are monotone 
increasing. The sequence {2 "} is strictly monotone decreasing. The sequence 
{(-1)"} is not monotone. The sequence {a„} = {1,-1, 1,-1,. ..,1,-1, 2, 4,6,8,...} is 

> V ' 

1000 

strictly monotone increasing for sufficiently large n {n> 1000 ). 

The next result is useful in numerous applications. 

Theorem 3.7.1. A monotone increasing (decreasing) sequence which is bounded 
above (below), is convergent. 

Proof. 

Consider a monotone increasing bounded above sequence {«„} and let M denote 
its least upper bound. Given an arbitrary f > we can find an element «„ of the 
sequence which satisfies M - e <a^ . Due to the sequence's monotonity we also 
have M -£ <a^ , n> n^ and since a,^ < M for all n , we obtain 
|fl„ - mI < f , « > «o . Since this is valid for arbitrary £■ > , we obtain 
lim a,^=M . The proof for decreasing sequences is similar and is left for the 

reader. 

( iV 

Example 3.7.2. Consider the sequence a„=lH — , n>l . By using the 
binomial theorem we get 

n{n-l)---(n-k + l) 1 



<-z;;i!hz 



i=ov^J U; t^ k\ n' 
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and consequently 



— zi'4' 



k=l 



kl[ nk n 



Similarly 



fl„a.l = 1 + 

k=l 



IL I ]L 2 ] L k-\\ 1 



-i-^-ZJi-„+i i-„+i -i-^+i + „^,.«.i 



(« + !)" 



and since 1 < 1 , 1 < 7 < A: - 1 we obtain «„ < a„^.i , i.e. the sequence is 

n n + \ 

monotone increasing. Also, a„ <l + lH 1 I-...H — (why?) and by using the 

2! 3! n\ 

inequality — < , m > 2 we get 

n\ 2 

.11 1 .11- (1/2)""' . 
a„<2 + - + — + ... + - = 2 + ^-^-^ — <3. 

2 2^ 2""' 2 1-1/2 

Thus, the sequence is bounded above and by virtue of Theorem 3.7.1 it is 
convergent. The limit of the sequence is denoted by the letter e (after the famous 
Swiss mathematician Leonard Euler (1707 - 1783), who was the first to use it as 
the base of the natural logarithm). 

f iV 

It is left for the reader to show e = lim 1 + — = lim b^ where 

b„=2 + — + — + ... + — ,n>2 
2! 3! m! 

The number e is transcendental, i.e., e is not a root of any polynomial with 
integer coefficients. The proof of this fact is complicated and is beyond the scope 
of this book. A proof that e is irrational, is much simpler to obtain but is 
postponed until we introduce the basics of Infinite Series. 

Example 3.7.3. Consider the sequence {a„} = {q"} where 0<^<1. It is 
monotone decreasing and bounded below by 0. Therefore, it converges to some 
A > . Now, define {b^ } = {q"'^ } . This sequence clearly converges to A as well. 
On the other hand it may be observed as the product of two sequences: [q"} with 



two 
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limit A and {q,q,...,q,...} with limit q. Thus, qA = A which implies 
(q - 1)A = . Since i^- ^^^ 1 we must have A = , i.e. lim i7"=0,0<i?<l. 

Example 3.7.4. Does the sequence «! = 1 ; «„+] = J2 + a„ ,n>\ converge? Let 

us examine monotonity, using induction. Since Aj = "v 3 we have Gj < 02 • 

Assume a„ < a,,^] . Then, a^^j = V ^ + fl«+i - V ^ + '^n = '^n+i which implies that 
{a„ } is monotone increasing. Is the sequence bounded? We will show a^<3 . 
This clearly holds for a^ . Assume a„ < 3 . Then 

'^n+i -yj^'^'^n <a/2 + 3 =v5 <3. Thus {fl„} is bounded by 3. Consequently, 
lim a„ exists. Denote it by A . Then, by using basic limit theorems we get 

lim fl^^j = /2 + lim a,, , which implies A = v2 + A or A -A-2 = 0. The 

n->oo v n— >~ 

solutions are A] = 2 and A2 = -1 but A2 is obviously exclude (why?), i.e. 
lim a„ = 2 . 

As previously stated, any sequence that does not converge is called divergent. 
There is however, a particular class of sequences, that do not converge in the usual 
sense, i.e. they do not satisfy the requirements of Definition 3.4.2, yet, they are 
still referred to as 'convergent sequences'. Instead of converging to finite numbers 
they converge to 'infinity'. In the next section we introduce the notations of 
'convergence to infinity' and 'infinite limits'. 

3.7.2 Convergence to Infinity 

Definition 3.7.3. A sequence {a„ } converges to infinity if for every M eSi there 
is n^e N , such that a^^ > M , n> Hq . We then write lim a„ = 00 (or 

lim a„ = +00 ). The sequence converges to minus infinity, if for every M eSl 

there is HqE N , such that fl„ < M , n> nQ. In this case we write lim a„ = -00 . If 

a sequence does not converge to a finite limit and also does not converge to +00 , 
the sequence oscillates. 

Theorem 3.7.2. An arbitrary monotone increasing sequence, which is not bounded 
above, converges to 00 . 

The proof of this result is left for the reader. 
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Example 3.7.5. The sequence {V"} converges to <x> . The sequence 
converges to oo as well (why?). Less trivial is the behavior of 



M^+3 



[«VM + « + ij 

n . 

{a„} where a„ =^— . Clearly, the sequence is strictly monotone increasing, 

k=i ^ 

and each term a^ , where m = 2" , can be represented as 



a„=l + - + \- + -\ + \ - + - + - + - \ + ... + \ i + i + ... + — 

2 l3 4J l5 6 7 8J U +1 2'""' +2 2"' 



,12 4 2'"-^ , m 

>1 + — + — + — + ... + = 1 + — 

2 4 8 2™ 2 

Thus, the sequence is not bounded above and by Theorem 3.7.2 must converge 
to oo . 

Theorem 3.7.3. Let {«„ } , {i>„ } be two sequences such that lim a„ = °° and 
a„ < b^ for sufficiently large n . Then, lim b„ =oo , 

The proof is straightforward and is left for the reader. 

Theorem 3.7.4. Consider a sequence {a„} where a,^>0,neN. Then, 
lim a„ = °o if and only if lim — = . 



Proof. 

Assume lim a„ = oo . Given an arbitrary e >0 define M = — . Then, find Hq 



such that a^> M for m > Mq . Thus, a„ > — or 



1-0 



1 



<e for M > Mg > 



which implies lim — = . The opposite is just as simple and is left as an 

exercise for the reader. 

Sometimes it is important to determine the rate by which a sequence converges 

to infinity. For example, the sequence {v«} 'seems' to converge to oo slower 
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than {n } . The sequence {n } 'seems' to converge to o° faster than {«} . What 

2 2 

can be said with regard to {n } vs. {2m } ? The following definition provides the 
exact definition of convergence rate. 

Definition 3.7.4. Let {a^ } , {^„ } be two sequences such that lim a„ = lim b^ =o° . 
If lim — ^ = « > , then the sequences are said to increase at the same rate. If 

lim — ^ = oo , then {fl„ } increases at & faster rate than {i>„ } . If lim — ^ = , then 
{a„ } increases at a slower rate than {&„ } . 

Note that if several elements of {/?„ } are zero, Definition 3.7.4 is still meaningful. 
Indeed, since b^ > , n > Hq for some fixed Mq , we simply disregard a„,/7„ for 
n<nQ as they do not have any impact on the limits or the rate of convergence. 

Example 3.7.6. The sequences [n } and [In ] increase at the same rate since 

,. n 1 „ „, , . \\n +n + \ . . ^. . 

lim — r- = — >0. The sequence \a ] = < 1^\ mcreases to mfmity at a 

n^^2n^ 2 I n + lOsIn I 

faster rate than {b^} = {yn} . Indeed, by using the basic limit theorems we get 

b a 

lim — ^ = . Therefore, by Theorem 3.7.4 lim — ^ = ^ , i.e. {a„ } increases faster 

«^~ fl„ «^~ b„ 

than {bj. 

The next result is another useful tool in determining whether an arbitrary 
sequence converges. 

Theorem 3.7.5. Consider a sequence {a,,} with nonzero elements such that 

= A (3.7.1) 



*n+l 



lim 

Then 

lim a„ = if A<1 



(3.7.2) 
lim|a„| = <=o if A>1 
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(a) Let A < 1 . For e ■ 



\-A 



< 1 we can find rin such that 



A - |a„+i /a„ I < £■ , M > Mq . Thus, |a„^.i /«„ I < A + f : 



1 + A 



q <l for n> riQ. 



In particular, a,j ^j < 'zhn +i ^^d by using induction we obtain 



k-i 



\'^n +k\^1 r« +1 ' ^ - ^ • 'Si'i'^^ < i^f < 1 we have lim «„ = . 



I 1 + A 
(b) Let A > 1 . In this case we get fl„^.i/a,J > = q>\ for n> iiq and since 

lim q" =oa we can apply Theorem 3.7.3 and obtain lim|a„| = oo . 



Example 3.7.7. Let a„ = — ,neN where r > , ^ > 1 . In this case 

q" 



lim 



*n+l 



:lim^-"^>^f =Umifl.^ 

n^°° M q 



? V ''/ "? 



<1 



n n 

which implies lim — = . Next, consider the sequence a„ = — ,neN. Here 
«^°= q" n\ 



lim 



*n+l 



lim^"^^^ -' = limfl.^ 



:e>l 



and consequently, lim — = oo , 

«->»= n\ 

We previously introduced the limit point notation and examined basic 
properties of such points. We now present several results related to the existence 
of such points. We will distinguish between a set whose elements are mutually 
distinct and a sequence for which this property is not necessarily valid. 

Theorem 3.7.6 (Bolzano- Weierstrass theorem for sets). Every infinite bounded set 
S has at least one limit point. 

Proof. 

Since S is bounded, we can find a closed interval [fli,^iJ which contains all the 



points of S . Define Cj 



a, + 1 



Then, at least one of the intervals 
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[ai,cj, [ci,iiiJ must contain infinitely many points of S . Denote it by [a2'^2J 
and define C2 =— -■ Again, at least one of the intervals [a2,C2J , [c2,i'2J 

contains infinitely many points of S . Using induction, we thus create a set of 
intervals {[fli,Z7iJ,[a2'^2J'---'l'^«'^nJ'---} such that {«„} and {Z?,,} are monotone 
increasing and decreasing sequences respectively, {a^} is bounded above, {b^} is 
bounded below and each interval contains infinitely many points of S . Therefore 
lim a„= A , lim b^=B for some real numbers A and B , but since 

< fc„ - a„ = — ' ' we get A = B . Finally, A is a limit point of 5 . To prove 

it we need to show that every interval (A-£,A + £) , £ >0 contains a point a 
such that ae S and a ^ A. Indeed, given an arbitrary £ >0 one can find Mj , «2 
such that A - f < fl,j < A ,n> n^ and A<b^ < A + £ ,n> 112- Choose 
Mq = max(Mi , M2 ) ■ Then A-f<fl„<A<i>„<a + £' for n>nQ. Consequently, 
the particular interval [a^^j,b^\,m = nQ+l is inside (A-£,A + £) and contains 
infinitely many points of S . Let a ^ A be one of them. Then la - AI < £■ and the 
proof is completed. 

A similar result can be derived for sequences. 

Theorem 3.7.7 (Bolzano-Weierstrass theorem for sequences). Every bounded 
sequence {a„ } has at least one limit point. 

Proof. 

What we have to show is the existence of a subsequence that converges. If some 
point a appears infinitely many times in the sequence, i.e., if there is a 
subsequence {a„ } such that a,, = a , k>l , then a is a limit of this 

subsequence, and hence a limit point of the whole sequence. If there is no such a , 
construct the following subsequence of {«„ } . Let Mj = 1 and find M2 > «i such that 
a„ 5^ a„ . This is possible or else «„ appears infinitely many times. At each step 
we can find m^.^) > m^ such that a„ '^ fl„ , 1 < / < A: , or else one of the numbers 
a„ ,l<l<k appears infinitely many times. We thus obtain a subsequence {a„ } 

with mutually distinct elements. This subsequence is therefore a set, and by virtue 
of the previous theorem it has, as a set, at least one limit point a . Namely, there is 
an infinite number of mutually distinct elements a^ ,a^ ,---,a„, ,-■■ such that 

a^ e {a„ ], />1 and lima,,, =a. Since there is no guarantee that 

ffl ; ' III. ' ' -J , tH I fJ 

nil <^2 < ■■■<ni: < ..., the number a is not yet a limit of subsequence of {«„ } . 
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The final step is therefore, showing how to construct a subsequence of 
a^ ,a^ ,...,a^ ,... (which must also converge to « - why? ) which is also a 

subsequence of {a^ } . We leave this step as an exercise for the reader. 

In the next subsection we discuss the derivation of properties of a sequence 
{a„} , using information about la^ -«„! ' where n,m are sufficiently large. 

3.7.3 Cauchy Sequences 

Definition 3.7.5. A sequence {a^ } is called a Cauchy sequence, if for an arbitrary 
£■ > we can find HqE N (which depends on e ), such that 

\^n~'^m\<^ ' n,m>nQ (3.7.3) 

Namely, a Cauchy sequence is characterized by the fact that for an arbitrary 
£■ > , as small as desired, there is a class whose members are almost all the 
sequence's terms, and the distance between each two members is less than e . 

Theorem 3.7.8. A Cauchy sequence {a^ } is always bounded. 
Proof. 

The choice e = \ guarantees the existence of some Hq such that |a„ - a,„ I < 1 as 
long as m,n> Hq. Thus, if we take m = n^ +1 we get \a„ - a,^ ^A< 1 , n > iiq . By 
virtue of the triangle inequality, this yields |fl„ I < 1 + a,, ^j , « > Mq . Therefore 

\a„\ <maxifl||,|fl9|,..., a„ ,1+ a„ ,, f, « >1 

and the sequence is bounded. 

The following result presents the equivalence between the notations of a 
Cauchy sequence and a convergent sequence. 

Theorem 3.7.9. A sequence {«„} converges if and only if it is a Cauchy 
sequence. 

Proof. 

(a) Assume lim a^ = A and consider an arbitrary fixed e >0 . Define f j = e/2 . 
Then, we can find Hq such that \a„ - A\ < e^ , n > Hq . Let m,n> iiq. Then 
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|a„ -a„| = |a„ - A + A-fl„| < |fl„ - A| + |A-fl,„| < 2f[ = f , in,n>nQ 
which implies that a convergent sequence is also a Cauchy sequence. 

(b) Let {a„ } be a Cauchy sequence. It is therefore bounded and thus, by Bolzano- 

Weierstrass theorem for sequences, {«„} has a limit point A . We will show 

lim a^ = A . Indeed, there is a subsequence {a„ } such that lim «„ = A . Let 

n— >~ '■ A:— >oo ' 

e>0. Find Mj such that |a„ -fl,„| <£■ , M,m >Mj and find k^ such that 
a„ - A < £■ , k > kQ. Let Mq the first n/, ,k>kQ , greater than «[ . Then 

I I Tl 

for all ne N . Since Mq > "i then n> Hq implies m,«o > «] and consequently 
a„ - a„ < £■ . Also, since Mq = "/ for some l> kf,, we have «„ - A < £■ . 
Therefore, \a^ - A\ < 2e , n > Hq , i.e., lim fl„ = A and the proof is completed. 

Example 3.7.8. If we replace the requirement given by Eq. (3.7.3) by a weaker 
requirement such as |fl„+i - «„ I < f , m > Mq > we may get a sequence that does not 

" 1 
converge. For example, the sequence a„ = /^— is divergent although it satisfies 

|a„+i -aJ = — and lim — = . 

n «->o° n 

Definition 3.7.6. A sequence {a„} is called a contractive sequence if 

K+i-«„|<?|««+i-«„|'«e A? 
for some 17 : < ^ < 1 . 

Example 3.7.9. Consider a sequence {a„} such that fl„^.i =2 + —^ . In this case 

we have a^^j ~'2«+i = 2+ ""*"' -2 ^ = — (fl„+i —a„), i.e. a contractive sequence 

with q = l. The conclusion is the same for a^^^ = A + qa^ ; A e 9? , (7 e (0,1) . 

Theorem 3.7.10. Every contractive sequence is a Cauchy sequence. 
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Proof. 

Let {a^} be a contractive sequence. Then, for every n e TV we have 

K+2 - ««+i I ^ ?|«„+i - a„ I < . . . < ?" |a2 - «i I 
In order to evaluate |fl„ - a,,, I we will assume m> n and rewrite 

|«m - «« I = |«m - «m-l + «m-l " • • • + ««+2 " ««+! + ««+! " «« | 

^ K« - «m-l I + K„-l - «m-2 1 + • • • + K+2 " ««+l | + K+1 " «« | 

< g'"^ 02 ~'^1 + 9™" P2 "«!+••• + "j'^K "'^l M" l"^ rl ~ '^U 

A m—n 

= q"-\l + q + ... + q'"-"-')\a2-ai\ = q"-'^^ k-aJ 

l-q ' 

n-\ 

< \ao — a,\ 

Since \iniq"~ =0, given f >0 there is Mq such that (7"" loj -aJ <^ for 

n>nQ. Consequently, |fl„-fl„J<f for m>n>nQ which implies that {a„} is a 
Cauchy sequence. 

Corollary 3.7.1. Every contractive sequence is convergent. 

The proof follows from Theorems 3.7.9-10. 

Theorem 3.7.11. Let {a„} be a contractive sequence which converges to A and 
has a contractive factor q . Then 

n-l 

fl„ ~ ^^ ^2 —cii\,ne N 

1-q' 

Proof. 

We previously obtained (proof of Theorem 3.7.10) \a^ ~<^n\ < r2 "''i fo"" 

1-q' 

n-\ 

fixed n and all m>n. Therefore, lim|a,„ -a„| = |A-a„| <-^ la, -a, I as 

ffi // 11 \ -t \ A, 1 

m->=' 1 — ^ 

Stated. 
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This theorem provides a useful tool for bounding (for a contractive sequence) the 
distance between a given a,j and the limit A . The bound depends on the 
contraction factor and the distance between the first two terms of the sequence. 



Example 3.7.10. Let a^^j = Jl + a^^ , a^ = 1 . The sequence is monotone increasing 
and converges to A = (why?). Also 



q«+2 - a«+i = Vi + ««+i - Vi + «« = L "^' — f^=- 



««+!-«« 



which, by Theorem 3.7.11, implies 



1 + V5 



2 
l + ^/5 



V2-I ^ 

For 



(2V2)""'(l-l/2V2) 



'10 



example, if « = 10 we get 

11 ujo iiiu IS about 1.74-10 

bound. 



< 5.53 -10 . The actual distance 



between Ajq and is about 1.74-10 , i.e. about a third of the calculated 



Theorem 3.7.12. Let {a„} be a contractive sequence which converges to A and 
has a contractive factor q . Then 

l-q' 

Proof. 

The proof is based on the previous technique (Theorems 3.7.10-1 1) and is left for 
the reader. 

Example 3.7.11. Consider the sequence from the previous example. For n = 10 

we get the bound — — (ajq - Qg) = 2.13 - 10" ( = means about or approximately) 
l-q 

which is a better estimate for A - Ajq than before. 



PROBLEMS 

1. Let fl„^.i =l + .^fl„ ,fli=l. Show that the sequence {a„}is monotone 
increasing and bounded above. Find its limit. 
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2. Let a„+i = — - — , Oi = 2 . Show that the sequence {fl„}is monotone 

decreasing and bounded below. Find its limit. 

3. Let a^^i = a^j + yja„ , Aj = 1 . Show that the sequence {fl„}is monotone 
increasing but does not have a limit. 

4. A sequence [a^ } which is monotone for n>nQ is called eventually 

monotone. Show that the following sequences are eventually monotone and 

n^ 3" 3" 

determine their type of mono to nity: (a) — (b) — r (c) 



2" n^+n-l in + n\ 

" 1 

5. Show that the sequence «„ = 2,^~ 1^^^ ^ limit A which satisfies 1 < A < 2 . 

k=i ^ 

6. Let {a„} and {b^} denote two monotone increasing sequences. Show that 
{c^} = {a^ +b^] is monotone increasing. Is {d^J = [a^ -b^} monotone 
increasing? 

^ , l-3-5---(2«-l) , 2-4-6---(2m) ^^ ^^ 

7. Let a„ = ^^ , b„ = ^ — — , ne N . Show: 

2-4-6---(2n) l-3-5---(2« + l) 

a. lim fl„fe„ = (b) lim fl„ = (c) lim b„=0 

„ _, , \n —n +n . ^ . 

8. Show that a,, = -7=— converges to inimity. 

9. Show that «„ = — converges to infinity. 

n\ 

10. Let {a^ ] , {i>„ } be sequences that satisfy lim a„ = and lim b„ =oo , What 

can be said about the sequence {c,, } = {a^^b^ } ? 

11. Change Eq. (3.7.3) and request |fl,,j-a^|<f for n,m>nQ ,\m-n\<k where 

^ is a prefixed positive integer. Is the sequence necessarily convergent? 
Explain! 

12. Let lim a„ = A . Show that the sequence of the averages converges to A , i.e. 



«! +(32 +••• + «„ . 



13. Prove lim — iH 1 I-...H — 1 = (Hint: First show monotonity). 

«->~ ny 2 3 n , 

14. Show lim(l + « + «^)'/"=l. 
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In this chapter we focus on a special class of functions called continuous 
functions. Basically, a continuous function y = f(x) is characterized by the fact 
that it consists of no jumps, i.e., a small change in X causes a small change in y . 
An exact definition of continuity is given only in Section 4.2. Until then we 
suggest the phrase given by some unknown mathematician: "When you draw a 
continuous function, the chalk never separates from the blackboard". 

4.1 Limits of Functions 

Definition 4.1.1. A function f(x) , xe D c 9t is said to have the limit A at a 
(or, "as X approaches a ") if: 

1 . a is a limit point of D . 

2. For every £■ > we can find a <5 > such that 

0<\x-a\<S ,xe D^\fix)-A\<£ (4.1.1) 

In this case we write lim f{x) = A. 

Note, that a does not have to belong to the function's domain. Also, the 
requirement that a is a limit point of D , implies the existence of a sequence 
{jt„ } , x„ G D with mutually distinct elements such that lim x„ =a and by virtue 

of the second requirement we have lim/(x„) = A. This assertion is left as an 

exercise for the reader. 

Example 4.1.1. Let f{x) = x , xe3i . Clearly, every real number is a limit point 
within 3i . We will show that the limit of f(x) at arbitrary jcq e 9t is Xg . Thus, 
given an f > we will find a ^ > , such that Ix - XqI < (5' implies \x^ - Xq\< £ . 
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Simple algebra yields 

I 3 3| IC \( 2 ^ ^ 2\^\ |/| |2_^| II I I |2) I |/| II |\2 

\X — Xq\ = \[X — Xq )\X + XXq + Xq \ < \X — XqKXI + \x\\Xq\ + \Xq\ <\X — Xq\\\x\ + \Xq\) 

Since we are interested only at a close neighborhood of Xq , we confine ourselves 

to x: Ix - Xgl < 1 . Therefore Ixl < 1 + IxqI which leads to 



\x - Xq\ < \x - XqM + 2\xq\) 



If we also require that 



\X X(\ \ 



7 I ra 



we obtain \x —Xq\<£. Thus, the choice S = min 



(l + 2k|f 



guarantees 



\x- Xq\< S - 



3 3 

X Xq \ C 



Example 4.1.2. The function /(x) = sin(x) is defined for all xe5R. We will 
show that the limit of /(x) at arbitrary Xg e 9? is sin(x0) . Indeed, 



TT 



sin(x)-sin(xQ) = 2sin[(x-XQ)/2]cos[(x + XQ)/2] 

Simple trigonometry implies that sin(a) < a < tan(a) for all <a < 

z 

I I ^ 

(see Section 5.3). Thus, by confining ourselves to x-Xq < — and using the 

inequality |cos(a)| < 1 which holds for all a , we get 

|sin(x) - sin(xo)| = 2|sin[(x - Xo)/2]||cos[(x + Xo)/2]| < Ix - XqI 
The rest is straightforward. 

In Examples 4.1.1- 4.1.2 we dealt with a particular case where the limit of /(x) 
and the value /(xq) exist. We applied a simple principle: Consider a function 
/(x) , XE D c 9^ and let x^e D be a limit point of D . If there is a constant 
p>0 such that the requirement 

xe £),|x-Xo|<yC)^|/(x)-/(xo)|<C|x-Xo| (4.1.2) 

holds, then lim /(x) = /(xq) . The proof is simple: Given an arbitrarily small 

e>0 define S = nnn{p,£/C}. If Ix - Xgl < <J then 
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|/(x)-/(xo)|<C|x-Xo|<C- — = £• 

which concludes the proof. Thus, if Eq. (4.1.2) holds even for a very small but 
prefixed p , the limit of f{x) as x^> x^ exists and equals /(xq) . 

Example 4.1.3. Let f(x) = sin(l/x) , x^O . The function does not have a limit at 
x = . Indeed, it is equal to 1 and -1 at x„ = and y„ 



(2n + l/2);T (2« + 3/2);r 

respectively, while both sequences of points approach as « increases. 

PROBLEMS 

1. Show that the function f{x) = cos(x) has a limit at all x : -co < x <o° . 

2. Show that the function f(x) = -Jx has a limit at all x.x>0. Does it have a 
limit at x = -1? 

3. Consider the function 

., , Jx^x^±l 

[ 0,x = l 

Does the function has a limit at x = -l,0,l? What are the differences between 
the three points? 

4. Let f{x) = xsin(l/x) , x^O . Does the function has a limit at x =0? 

5. Let f{x) = . For which X the function has no limit? 

(x - a)(x - b)(x - c) 

6. Let 

1 + V^,x<0 



sinl — I , X > 



fix)- 



Does /(x) have a limit at x = ? If not, can you redefine it so that the limit 
will exist? 

4.2 Continuity 

The class C of continuous functions is a very small segment of the set of all 
functions but is nonetheless very useful. The reason is that most processes in 
nature can be simulated using continuous functions or functions that are 
continuous almost everywhere. Actually the most applicable functions in the 
various sciences are even a smaller subset - functions that possess derivatives. 



90 4 Continuous Functions 

These will be treated in detail in Chapter 5. In this chapter we concentrate on the 
concept of continuity and the properties of the continuous functions. 

Definition 4.2.1. A function f{x) , xe D c 9t is called continuous at x = ae D 
if a is a limit point of D and if 

lim/(x) = /(fl) (4.2.1) 

If a is an isolated point in D , the function is not continuous at a . A detailed 
version of this definition would be: for arbitrarily small f > we can find S >0 
such that 

\x-a\<S ,xe D^\f(x)- /(a)| < £ (4.2.2) 

If f{x) is not continuous at x = a it is called discontinuous at this point. 

The domain D of f(x) is usually an interval, closed, open, semi -infinite, the 

whole real line or a union of such intervals. However, Definition 4.2.1 considers 
the most general domain D . 

Example 4.2.1. Let f(x) = 2x over the domain 

D =[-l,0]u{l,l/2,l/3,. ..,!/«,...} , composed of a single closed interval and a 
sequence of points which converges to from the right. It is easily seen that the 
function is continuous over the interval, including at , and is not continuous at 
each x„ = l/n which is an isolated point. 

Definition 4.2.2. A function f(x),xsDc3i is called continuous over D if it 
is continuous at all x: xe D . 

Example 4.2.2. The functions x ,sin(x),cos(x) are continuous for all 
X : -o° < X <o° . This is derived using the same algebra as in the previous section 
and is left for the reader. 



Example 4.2.3. Let 

fix)- 



\x^ ,x^+l 
0,x = +l 



The function is not continuous at x = +1 . However, by altering its definition, the 
continuity may be extended to hold there as well. This can be done by simply 
redefining/(l) = /(-!) = !. 

Some discontinuities can never be removed. For example the function 
/(x) = sin(l/x) , X ;^ oscillates between 1 and -1 as we approach x = and 



4.2 Continuity 91 

no matter how we define the function at this point, it is impossible to obtain 
continuity since the basic requirement of Eq. (4.2.1) is not satisfied, simply 
because the left part of that equation does not exist, i.e., the function does not 
converge as x — ^ . 

Another type of irremovable discontinuity occurs when a function is not 
bounded. For example, the function 

f 1/x ,xi^Q 
\A,x=0 

is not continuous at x = for all finite A . The proof is left for the reader. 

A removable type of discontinuity occurs only in the case when the left-hand 
side of Eq. (4.2.1) exists. Then, if f{a) is not defined we may take 

/(a) = lim/(x) 

and if the function is defined but satisfies f(a) ^ lim/(x) we simply change and 
redefine f{a) = lim/(x) . This last case is called jump discontinuity. It disappears 
via redefining the function. 

Example 4.2.4. Consider the function 

1 , X rational 



fix)- , 

[0 , X irrational 

This function is discontinuous for all x , rational and irrational alike, since at any 
arbitrarily small interval there is always at least one point of each type. The exact 
details are left as an exercise for the reader. 

Example 4.2.5. Let each rational number be represented as m/n where m,n have 
no common divisor. Define 

\l/n , X = m/n 
fix) = \' ' 

yd , irrational 

The function is clearly discontinuous at all rational points since at arbitrarily small 
interval about any rational number r there is at least one irrational number where 
the function vanishes. This contradicts the requirement of Eq. (4.2.1) since the 
function's value at r = mjn is Ijn ^ . On the other hand, at an irrational point 
^ , the function is zero and for a given e>Q we can find a sufficiently small 
interval [^-S,^ + d] at which the function values at the rational points within 
this interval, are less than £ . All we need to do is to choose an interval that 
contains only rational numbers m/n for which n > l/e (why is that possible?). 
Consequently, f{x) is continuous at ^. 
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Since the domain of definition of a given function is quite often an interval, we 
introduce the notation C[a,b]- all the continuous functions defined over a closed 

interval [a, i>] . Similarly we define C(a,b) etc. 

Definition 4.2.3. A function f(x) is said to be right-continuous at a if 

lim/(x) = /(«) (4.2.3) 

where by x — > a"*" we mean x = a + h,h>0,h—>0. Left continuity is defined 
similarly. 



Example 4.2.6. Let 

fix)- 



X , X < 1 

2,x>l 



The function, illustrated in Fig. 4.2.1, is defined everywhere, continuous at x # 1 , 
has a jump continuity at x = 1 but is left-continuous there. The discontinuity is not 
removable since the left and right limits are different. 



}'=/(-i) 



(1,1) 




Fig. 4.2.1. The function of Example 4.2.5 - left continuity at x = 1 . 

Theorem 4.2.1. If /(x) is right and left continuous at an arbitrary point x = a , 
then /(x) is continuous at a . 

The proof is left as an exercise for the reader. 
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Theorem 4.2.2. A necessary condition for the continuity of f(x) at an arbitrary 
X = a , is that f(x) is bounded, i.e., there exist S >0 and M :0< M <o° such 
that 

\x-a\<S^' |/(x)|<M 

Proof. 

If f{x) is continuous at a but not bounded there, we can find Xj within the 
interval /i=[fl-l,fl + l] such that |/(xi)| > 1 , otherwise f(x) is bounded in /j 
which contradicts our assumption. Next, we can find Xj within 
/j = [a -1/2, a + 1/2] such that |/(xi)| > 2 , otherwise f(x) is bounded at a . We 
are thus able to construct a sequence {x„ } for which a - l/n < ^2 < a + 1/m and 
|/(x„)|>M. However, while limx„ =a, the values If(x^)} approach infinity 

rather than f(a) , which contradicts the continuity assumption. 

PROBLEMS 

1 . Show that the following functions are continuous for all x : -o° < x <oo ; 

(a) fix) = x^+x (b) f(x) =1 X I (c) f(x) = l/[2 + sin(x)] . 

2. Consider the function 



f(x)- 



0,x = 

3,x = -l 
sin(l/x) + cos[l/(x + l)],x^ 0,-1 



Find the points of discontinuity. Is it possible to remove these discontinuities 
by redefining the function? 

3. Let f(x) , a< x<b satisfy the Lipschitz condition \f(x) — f(y)\ < L\x- y\ 

for arbitrary x,y : a < x,y <b . Show that f{x) is continuous over [a,b] . 

4. Let f(x) be defined everywhere, satisfy f(x+y) = f(x) + f(y) for all 
X, y and continuous at x = . Show that /(x) is continuous everywhere. 

5. Show that /(x) = —j^=^ is continuous for all x > 0. 

Vx + 1 + -J X 

6. Find the points of continuity and discontinuity of 

V — X , — oo < X < 
/(x) = <! 1/1000 , < X < 1 

1/1000+ Vx-1 ,l<x<oo 



x\ 


-oo< x<l 


(x- 


1), 


l<x<10 


1/8 


,10 


<X<t>o 
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7. Find the points of continuity and discontinuity of 

4.3 Properties of Continuous Functions 

Throughout the rest of this chapter we mainly deal with the set of all continuous 
functions. Our next result shows that continuity is preserved under the four basic 
arithmetic operations. 

Theorem 4.3.1. Let f{x),g{x) denote arbitrary continuous functions defined 
over the same domain. Then, f(x) ± g{x) , f(x)g{x) are continuous as well, and 
f{x)/g(x) is continuous at all x for which g(x) ^0 . 

Proof. 

The proof is derived in a similar manner to that of Theorem 3.5.1. The proof of the 
first three claims are left for the reader while that of the fourth will be given here 
in detail. 

Define H(x) = f{x)/g(x) . Assume g(a) ^ and let e >0 . We will show the 
existence of ^ > such that 



Clearly, 



H(x)-H(a) 



x-a\<S^> \H(x)- H(a)\ <e 

fix) f(a) _f(x)g(a)-f(a)g(x) 



Hence 



(x) g(a) g(x)g(a) 

f{x)g{a) - f{a)g{a) + f(a)g(a) - f(a)g(x) 
g(x)g(a) 



,„, , „/ xi/ /W"/M \f(<^')\\8(x)-g{a)\ 
\H (X) - H (a)\ < j j - + - 



\g(x)\ \g(x)g{a)\ 

Since g{a) t- and g(x) is continuous, we can find (how?) Jj > such that 

I I i^ I / xi k^"^)! 

X - fl < Oi ^> Le(x) > 

The continuity of /(x) at a guarantees that given the particular choice 
£■[ = — we can find O2 > such that x - a < O2 ^> /(x) - /(a) < £"1 . Also, 
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£\gW\ 
due to the continuity of g(x) at a , if given the choice e^ = — r-i — ■ — ■ — rt , we can 

4[l + |/(«)|J 
find ^3 > such that 

|x-a| < ^3 ^> |g(x)- g(a)\ < £2 

Therefore, by taking S = min( Jj , ^2 ' '^3 ) we obtain 

' ' 4 /'I* I ' \g(af 4[l + |/(a)|J 

provided that Ix - al < <J . This completes the proof. 

Example 4.3.1. Since a constant function and the function /(x) = x are 

continuous everywhere, so is an arbitrary polynomial f{x) = Oq + a^x + . . . + a„x" 
which can be obtained from these two using addition and multiplication. 

Example 4.3.2. The function tan(x) = sin(x)/cos(x) is continuous except at 
x = {n + y2)7r where n is an arbitrary integer. Indeed, sin(x) and cos(x) are 
continuous everywhere and the denominator vanishes only at x = {n + y2)7r . 

Let g{x) denote a continuous, defined over an interval / and let the set 
S = {y\ y = g(x),xs 1} , i.e., the image of g{x) , be a subset of an interval J over 
which another continuous function f(x) . Clearly, the function f{g{x)) , called 
the composite function and denoted by (/ o g){x) , is defined over / . 

Theorem 4.3.2. The composite function (f ° g){x) is continuous over /. 

Proof. 

Given a within the interior of / , we have to show that for an arbitrary e >0 
there exists ^ > such that 

\x-a\<S,xe I=>\f(g(x))-f(g{a))\<£ 
The continuity of f(x) guarantees that for some ^j > we have 

\y - g(a)\ <S^,ye J ^ \f{y) - f{g{a))\ < £ 
Also, since g{x) is continuous over / , we can find S>0 such that 

X — a < <J,xe / => \g(x) — g{a)\ < Si 
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The rest is straightforward: 

\x-a\<S,x& I ^\g{x)-g{a)\<Sy ^\f(g{x))-f{g{a))\<e 
and the composite function is indeed continuous at a . 

The next result is one of several that relate to an arbitrary continuous function over 
a closed interval. 

Theorem 4.3.3. A continuous function /(x)over a closed interval I = {a,b'\ is 
bounded, i.e., there exists M > such that |/(x)| < M for all xe I . 

Proof. 

If f(x) is not bounded, then a sequence {x„} , x„ e / such that |/(x„)| > n , can 

be found. Since the sequence is bounded, then by virtue of Bolzano- Weierstrass 
theorem for sequences (Section 3.7), it has a limit point i^ . Thus, there is a 

subsequence {x„ } of {x,,} that converges to ^ . The continuity of f{x) implies 

lim /(x,j ) = f{^) which leads to contradiction since /(x,, ) >«^ — »oo. Thus, 

f{x) must be bounded which concludes the proof. 

By virtue of Theorems 3.2.1 and 4.3.3, a continuous function f{x) over a 
closed interval / = [a,b] has a supremum, i.e., there exists a number M such that 

fix)<M ,XE I (4.3.1) 

and for an arbitrary £>0 we can find x^ e I that satisfies f(x^)>M-£. 
Similarly, f{x) has an infimum m , i.e., a number for which 

f{x)>m,xel (4.3.2) 

and such that for an arbitrary e >0 we can find x^ e I that satisfies 
f{Xg) <m + e . 

The next result shows that these supremum and infimum are attained by the 
function, i.e., they are the maximum and minimum of f{x) , respectively. 

Theorem 4.3.4. Let f{x), a continuous function over the interval I = [a,b] , have 
a supremum M . Then, there exists a point ^ & I such that f{^) = M . 

Proof. 

Since M is a supremum of f{x) , we should be able to construct a sequence 
{x„},x„e/ such that f{x^)>M-l/n which implies that the sequence 
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{/(•^n)} converges to M . By applying once again the Bolzano-Weierstrass 
theorem for sequences, we construct a subsequence {x„ },k =1,2,... which 
converges to <^ e I ■ The continuity of f(x) at ^ imphes Um /(x„ ) = /(^) , but 

since Um /(x„ ) = M as well, we get M = f(^) which concludes the proof. 

Similarly, a continuous function over a closed interval attains its infimum, i.e. has 
a minimum. 

Theorem 4.3.5. Let f{x) denote a continuous function over the interval 
I = [a,b] with maximum and minimum M ,m respectively, and consider an 
arbitrary number c :m<c < M . Then, there exists at least one ^ : a< ^ <b such 
that /(a = c. 

The proof of Theorem 4.3.5 follows directly from the following lemma. 

Lemma 4.3.1. Let f(x) denote a continuous function over the interval / = [a,b] 
which satisfies f{a)<0,f{b)>0. Then, there exists at least one point 
^:a<^<b such that /^) = . 

Proof. 

Let c = {a + b)/2. If /(c) = we choose ^ = c. If /(c) <0 we define 
ay=c ,b^=b and if /(c) > we take a^= a ,b^= c . We continue the process 
with the interval [01,^11] where /(«;)< , /(Z?;) > and (Z?; - a; ) = (i> - a)/2 . 
Prior to the (« + l)-th step we have an interval [a„,Z7„] for which 

/(a„)<0,/(fe„)>0 and (fe„ -«„) = (Z7-fl)/2« . Define c = («„+fe„)/2. If 
/(c) = the process stops. If not, we have two cases: If /(c) <0 we define 
a„+i = c , fo„+i = Z7„ and if /(c) > the new interval for further search is 
a„+i = a„ , ^„+i = c . In either case we proceed to the {n + 2) -th step with 

/(a„„)<0,/(^„,i)>0 and (fe„^i -a„^i) = (^-«)/2""' ■ 

Now, by virtue of Bolzano-Weierstrass theorem for sequences, we can find a 
subsequence {a„ } of {a,, {which converges to some ae I . Since 

lim(fe„ -a„ ) = (why?) we get limi>„ =a as well. The continuity of /(x) 
implies 

lim /(a„^ ) = Im /(\ ) = /(«) 

but since /(a„ ) < , /(Z?,, ) > for all k we obtain f(a) = (why?) which 
completes the proof of the lemma. 
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The proof of Theorem 4.3.5 follows immediately: Let /(xj) = M and /(jcj) = m . 
If M =m the function is constant and the theorem's claim is trivial. Otherwise, we 
may assume m <c < M . The function g(x) = f{x) - c is also continuous over 

/ = [a,fc] and satisfies g(X[) = M -c > , gCxj) = m-c < 0. Therefore, by 
virtue of the previous lemma, there exists ^ between x, and X2 (therefore within 
[a,ft]) such that g(^) = /(^)-c = 0,or /(^) = c. 

The result of Theorem 4.3.5 implies that a continuous function f(x) over a 
closed interval I = [a,b^ defines a range Rf =[m,M] where M ,m are the 
maximum and minimum of f(x) respectively. 

PROBLEMS 

1. Let the function f{x) be defined and continuous over the interval [a,b] (At 

the endpoints we assume one-side continuity). Show that g{x) = [f(x)]" is 
continuous over the same interval. 

2. Consider a continuous function f(x) such that f(x) > for all x . 

(a) Show that g(x) = ^ f(x) is continuous at all x = a such that f(a) > 
(Hint: rewrite V7W -^[^ = [f(x)- /(a)]/[V7W + V^] )■ 

(b) Show that g(x) = ^J fix) is continuous even when /(x) = . 

3. Explain why the function /(x) = sin[x +sin (x)] is contmuous everywhere. 

4. Find the domain of continuity for the functions: (a) ^sin(x) + cos(x) 

(b) l/(x'* - 1) (c) ^2-sm^ix) (d) ^ll/smix) + l/cos(x) . 

5. Show that continuity of f(x) guarantees the continuity of |/(x)| but not the 
opposite. 

6. Show that the function /(x) = sin(x) attains the value 0.55 somewhere 
within the interval [7r/6,7r/2] . 

4.4 Continuity of Special Functions 

In this section we present several results related to monotone and inverse 
functions. 

Lemma 4.4.1. Let a continuous function /(x) defined over a closed bounded 
interval [a,b] , satisfy f(a) < f(b) . Then, for every A , f{a) < A< f(b) there 
exists c:a <c <b (not necessarily unique) such that f{c) = A . 
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Proof. 

The function g(x) = f(x)- A is defined and continuous over [a,b] and satisfies 
g(a) = f(a) - A < and g(b) = f(b) - A > . By Lemma 4.3.1, there exists at 
least one c:a <c <b such that g(c) = f(c) - A = , i.e., /(c) = A . 

A repeated use of this lemma provides the next result. 

Theorem 4.4.1. Let a continuous function f(x) defined over a closed bounded 
interval [a,b'\ , obtain each value between f{a) and f{b) exactly once. Then, 
f{x) is strictly monotone. In particular, if f{a) < f{b) , then f{x) is strictly 
monotone increasing, and if f(a) > f(b) , f(x) is strictly monotone decreasing. 

Proof. 

Let for example f{a)<f{b) and assume /(xj)>/(x2) for some 
a<x^<X2<b. If f{x^)>f{a) as well, then by Lemma 4.4.1 each value 
between max{/(a),/(x2)} and /(xj) is attained at least twice, first at the 
interval [a,Xj] and secondly at [x^Xj]. This contradicts the basic assumption of 
Theorem 4.4.1. A similar contradiction occurs if /(xj) > f(b) . 

Next, assume f{a)>f(x^). Since f(a)<f{b) we get f(x^)<f(b) and 
consequently, by applying Lemma 4.4.1, each value between /(X2) and 
min{/(i>),/(x,)} is attained at least twice and the assumption of Theorem 4.4.1 
is again evoked. Thus, /(xj) < /(X2) (why not /(xj) = /(X2) ?), i.e., /(x) is 
strictly monotone increasing. The case f(a) > f(b) in which /(x) must be 
strictly monotone decreasing, is treated similarly. This completes the proof. 

The next result relates to inverse functions. 

Theorem 4.4.2. Let /(x) denote a continuous strictly monotone function over a 

closed bounded interval [a,b]. Then, the inverse function /" (x) exists and is 
continuous strictly monotone of the same type. 

Proof. 

Assume first that /(x) is strictly monotone increasing. By Lemma 4.4.1, for each 
yQ : f{a) < yo < fib) there is at least one Xq such that /(xq) = yQ . Since /(x) 
is strictly monotone increasing Xq is unique and inverse function of /(x) 
denoted by /" (y) , which assigns Xq to y^ exists. Let }'i,}'2 
satisfy f (a) < yi < y2 < f(b) and determine the unique Xj,X2 for which 



100 4 Continuous Functions 

/(xi) = ji and f(x2) = y2- Thus, f~\y^) = Xi and f~\y2) = X2. Clearly 
X[ ^ Xj , otherwise jj = jj in contradiction with the previous assumption. If 
Xi>X2 then }'i > }'2 , which evokes the assumption that f{x) is strictly 

monotone increasing. Consequently Xj < X2 , i.e., f(y) is strictly monotone 
increasing. 

The continuity of f^^(y) is shown as follows. Assume that /" (y) is 
discontinuous at yQ . Then, there exists a sequence of distinct points 
y^ , n = 1,2,... within the interval [f{a),f(b)] , such that lim y^ = yQ while the 

sequence x„ =/" (y^J does not converge to Xq = /" (jo) ■ Since /" (j;) is 
strictly monotone, all x^ are distinct and within [a,b] . Also, for some e >0 
there exists an infinite subsequence {x„ } such that \ x^ - Xq\>£ for all / . By 
Theorem 3.7.6 (Bolzano - Weierstrass theorem) an infinite subsequence of {x,, } 
converges to some Xq^Xq. Denote this final subsequence by {z,}. Since 
f{x^)^>yQ as «^oo, f(Zj)^>yQ as well and the continuity of f{x) 
guarantees /(Xq) = )'o- This contradicts the relation Xo=/"'()'o) since 
Xq ^ Xq. Thus / '(y) is continuous everywhere which completes the proof. The 
case of a strictly monotone decreasing f(x) is treated a similar manner. 

Example 4.4.1. Let f{x) = x ,0<x<10. The function is continuous and strictly 

monotone increasing. Therefore, its inverse g(x) = -Jx , < x < 100 is continuous 
strictly monotone increasing as well. 

Example 4.4.2. Consider f(x) = sin(x) , < x < ;r/2 . The function is continuous 
strictly monotone increasing and consequently, so is its inverse denoted by 
sin"'(x) (or frequently by arcsin(x) ) and defined over the interval [0,1] . 

The exponential function y = a^ ,a>Q is quite frequently used and taken for 

granted as continuous. We will now introduce it properly and show its continuity. 
The process includes four stages: (1) Defining an integer power and showing basic 
rules (Eq. (4.4.1) below); (2) Extension to rational powers; (3) Extension to 
irrational powers; (4) Establishing continuity of the exponential function. 

Definition 4.4.1. For arbitrary a > and a positive integer n 
a =l,fl =l,fl = a ■ a , n> I ; a = 1/ a 

Theorem 4.4.3. For arbitrary a > and positive integers m,n we have 
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fl^fl" =«'«+«, (a"- )«=«'«■« (4.4.1) 

The proof, using induction, is straightforward, and can be easily extended for all 
integers. It is left as an exercise for the reader. 

Our next result provides a proper definition to the n-th root of a positive 
number. 

Theorem 4.4.4. For arbitrary a > and a positive integer n the equation x" = a 
has a unique solution denoted by yja or a ' " . 

Proof. 

The continuous function /(x) = x" -a satisfies /(O) < , /(l + a) > (the 
second result follows from Bernoulli's inequality). Therefore (by Lemma 4.3.1), 
there exists Xj : < Xj < 1 + a such that x" = a . If Xj > Xj (xj < Xj ) then 

Xj > X[" (xj < X[" ) . Thus, Xj is unique. 

We now define powers with rational exponents. 

Definition 4.4.2. For arbitrary a > and positive integers m,n : 

a'"/«=(flV«)'« 

The next result is an extension of Theorem 4.4.3. 

An alternative definition for a rational power is fl™'"=(a'") . This is a 
consequence of the following result. 

Theorem 4.4.5. If a rational power is defined by Definition 4.4.2 then for arbitrary 
a >0 and positive integers m,n : 

a"'/"=(a'"f" 

Proof. 

Let X = (fl"")'" , y = (fl™)"" . Then, x"'" = a"" and y''=a'". The first relation 

provides (x"™)" =a . Thus, (x"'")"'" =0" and by applying Theorem 4.4.3 we 

obtain x" =fl™ . Therefore x" = y" and consequently x = y which completes the 
proof. 

Theorem 4.4.3 may now be extended to rational powers. 
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Theorem 4.4.6. For arbitrary rational numbers r, , rj and a > we have 

Proof. 

Let ^1 = pjq , ^2 = m/n . Then 1 + '2 = (pn + <ltn)/{qn) . Therefore, by 
Theorem 4.4.3, 

^'i+'2 — ^(pn+qm)l{qn) _r^l/(qn)-i(pn+qm) _ ^(pn)l{qn) ^{qm)l{qn) — ^n ^''2 

The proof of the second claim is a bit more complicated. We have to show that 
Clearly, (a"«>)"" = a"<^"^ (why?). Therefore, by Theorems 4.4.3 and 4.4.5 

and the proof is completed. 

We are now ready to define an irrational power of an arbitrary positive number a . 
We first confine ourselves to a >l . 

Definition 4.4.3. Let a > 1 and consider an arbitrary irrational number x . The 
power fl ^ is defined as 



a 



lima''- (4.4.3) 



where {r„,« = 1,2,...} is a decreasing sequence of rational numbers which 
converges to X . 

Clearly, this definition is meaningful only if the sequence {«''' ,n = l,2,...} 
converges and if the limit is independent of the particular sequence 
{r„,M = 1,2,...} . The first request is satisfied since the sequence {«'",« = 1,2,...} is 
monotonic decreasing and bounded below (for example by 0). Let {5„,n = 1,2,...} 

be another decreasing sequence that converges to x . Denote a'" -^ A , a'^" -^ B 
as n— ^oo. For an arbitrary fixed m there exists nQ{m) such that 

s^ < r^ , n> nQ{m) . Since a>l this yields a"" < a'"' , n > nQ{m) as well (why?). 

Consequently, B<a'"for all m . Thus, B<A. Similarly A<B which yields 

A = B . Needless to say that by defining a^ via a monotonic increasing sequence 
that converges to x , one gets the same value (this is left as an exercise for the 

reader). Thus, the power a^ is well defined for all real numbers x and a > 1 . For 

< fl < 1 we define a" = l/a"" and also l"" = 1 . 
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We now establish the result regarding the continuity of the exponential 
function. This is done in two steps. 

Theorem 4.4.7. The function a ' , a > 1 is strictly monotone increasing for all x . 

The proof can be designed by arguments similar to those used in the discussion 
added to Definition 4.4.3, and is left for the reader. 

Theorem 4.4.8. The function a^ ,a>\ is continuous for all x . 

Proof. 

(a) X = . We will show a^ ^ a as y ^0 . For arbitrary £■ > we must show 

the existence of S>0 such that \y\<S yields la' -ll<£'. This is done as 

follows. Since a " — » 1 , a" " ^ 1 as n — ^ oo , we can find «q sufficiently large 
such that 

\ a -l\< £ ,\ a -l\<£ , n> iiq 

However, since the exponential function is monotonic, for each 
y:-\ln<y<\ln we obtain \a^ -\\< e which concludes the continuity at 
x = 0. 

(b) x^O. In this case a'^^ -a'^=a'^a - a'^ = a'^ {a -1) and since 
limla -11=0 as shown in part (a), we obtain limla^""" -a^\=Q which 
concludes the continuity at xi^Q . 
PROBLEMS 

1. Let f{x) = 2x - x + \ ,\< x<2 . Prove that the function is strictly 
monotone increasing by showing that it uniquely attains every arbitrary 
value between /(I) and /(2) . 

2. Obtain the inverse function of f{x) of problem 1 if it exists. Is it 
continuous? 



3. Find the inverse of y = ^\ + x over x:Q< x<oo . 

4. Show lim a^ =0° for all arbitrary a>\ . 

71 

5. Show that y = cos(x) , < x < — has an inverse. Does the function 

2 

y = cos(x) , < x< — have one? 

2 2 
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4.5 Uniform Continuity 

We now introduce the concept of uniform continuity. A function 
/(x) , X e D c 9? is said to be continuous at ae D if for every e >0 , a number 

S>0 such that \x-a\<S,xeD imply \f(x)-f{a)\<£, can be found. 
Usually, S depends on both e and a . If, however, a continuous function /(x) is 
such that for an arbitrary £ >0 , the number S is independent of a , /(x) is 
uniformly continuous. An equivalent definition is given next. 

Definition 4.5.1. A continuous function /(x),xe D c 9? such that given an 
arbitrary e >0 , a number S{£) > which for all x,ysD satisfies 

|x-y|<^^|/(x)-/()^)|<£- (4.5.1) 

is said to be uniformly continuous. 

The main feature of uniform continuity is ^ 's independency of x,y . 

Example 4.5.1. Consider the function 

fix) = - , < X < 1 

X 

It is clearly continuous throughout its domain of definition, but is not uniformly 
continuous. Indeed 



f(x)-f(y) 



1 _ 1 

X y 



\xy\ 



■^' (4.5.2) 



and given an arbitrary £■> , the inequality \fix)-f(y)\<£ is equivalent to 
Ix- yl < e\xy\ . Assume the existence of S , independent of X, y for some given 
e >0 and let y = S,0<x<S. Then, Ix - ^1 < e\xS\ which leads to 
contradiction when x ^ . Consequently, /(x) is not uniformly continuous. 

Example 4.5.2. The function /(x) = x , < x < 1 is uniformly continuous. 
Indeed, let f > , < x, y < 1 . Then 

|x^ - y3U |x - jM +xy + y^\<3\x- y\ (4.5.3) 

g 

Consequently, the choice S = — guarantees 

\x-y\<S^\f{x)-f{y)\<e 
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i.e., X is uniformly continuous over (0,l) . 
Another, somewhat less trivial example is given next. 

Example 4.5.3. The function f(x) = ^jx , < x < 1 is uniformly continuous. Let 
e >0 ,0< x,y <l. Since Ix - yl = y-Jx + ■yjy) Nx - ■yjyl, the relation 

|/(x)-/(}')| = Ulx - -slyl < £ holds if and only if the inequality 

|x - y] < e[Jx + y[y) 

holds as well. However, sincel-Jx - ■yjy\< ^Jx + ^jy ,0< x,y <l, the choice 
S = £ , independent of x,y , implies 

\x-y\<8^>V[x-.y^\ <\Jx-.^h[x+^)=\x-y\<8 = e^ (4.5.4) 



\^-4y\ 



<e . 



The following result provides a sufficient condition for uniform continuity of an 
arbitrary continuous function. 

Theorem 4.5.1. A function f{x) continuous over a closed interval [a,b'\ is 
uniformly continuous over the interval. 

Proof. 

If f{x) is not uniformly continuous over the interval, an f > can be found such 
that for any arbitrary ^ > a pair of points x,y;a< x,y <b for which 

\f{x)- f{y)\>£ , exists. For each ^„ = (fo-a)/2" , m = 0,1,2,... let x„,y„ 
denote a pair such that 

Vn -yn\<Sn^ |/(^„) " /(^j] ^ £ (4.5.5) 

Since all the pairs are within the bounded interval [a,b] , converging subsequences 
K}'K1''*^ = 0,1,2,... such that 

lim x„^ = lim y„^ =A,0<A<B (4.5.6) 

exist. However, the continuity of f{x) at the particular point A implies 

lim /(x„^ ) = lim /(j.,,^ ) = /(A) (4.5.7) 
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and consequently lim[/(x„ )- f(yn )] = which contradicts Eq. (4.5.5). Thus, 
f{x) is uniformly continuous as stated. 

Theorem 4.5.1 can be easily extended to include continuous functions defined 
over an arbitrary closed set. This is left as an exercise for the reader. 

PROBLEMS 

1. Is the function f(x) = x ,0<x<l continuous? Is it uniformly 
continuous? 

2. Let /(x) = ,0<x<l. 

x-l 

(a) Is the function continuous? 

(b) Is the function uniformly continuous? 

(c) Explain the difference between this function and the function of 
problem 1 . 

3. Show that a function for which a Lipschitz condition holds, is uniformly 
continuous. 

4. Which of the following functions are uniformly continuous: 

(a) /(x) = Vx ,-oo<x<10 (b) /(x) = V^,-10<x<1000 

(c) fix) = - , 0.001 < X < 1 (d) fix) = siJ ^ , 0<x< — 
X {yUj 4 

5. Show that if f(x) is uniformly continuous over [a,b\ and over [b,c\ it 
is also uniformly continuous over [a,c\. 

6. Show that if fix) is uniformly continuous over {a,b) and {b,c) and is 
continuous at x = ii , it is also uniformly continuous over (a,c) . 

7. Which of the following functions satisfy a Lipschitz condition: 
(a)/(x) = x^-l<x<3 (b) /(x) = Vx,0<x<3 

(c) fix) = sinj - I, 0<x<— (d) fix) = Vx , < X < 1 
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In the previous chapter we investigated a special class of functions, C , the 
continuous functions. In this chapter we investigate a small subset of C , called 
differentiable functions or functions that possess derivatives. The concept of 
derivative, introduced by Newton and Leibniz, marked the beginning of modem 
calculus. Once familiar with the limit notation of Chapter 3, we are ready to define 
the concept of the derivative of a given function. 

5.1 A Derivative of a Function 

Consider a function y = f{x),a<x<b (Fig. 5.1.1) and two arbitrary points 
Xq,Xq+ Ax such that a < Xq,Xq + Ax <b . Clearly, the number Ax measures the 
change in x while moving from x^ to Xq+ Ax . Let Ay denote the change in y 
(the function's value), i.e. Ay = f{xQ + Ax)- f{x) . Then, the quantity Ay /Ax 
represents the average change (increase or decrease - depending on the sign of 
this number) in y over the interval [xq , Xg + Ax] . 




Fig. 5.1.1. First step in defining the derivative: average change. 
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The average change is also the slope of the secant defined by the points 
(xo,/(xo)) and (xg +Ax,/(xq + Ax)) . The behavior of the slope as Ax — > is 
one of the most basic concepts of calculus and is treated by the next definition. 

Definition 5.1.1. Let f{x) be defined at some neighborhood of Xg . The 
derivative of f(x) at Xq is 

/(xo+Ax)-/(xo) 



/'(xo)=lim 

Ai^o Ax 

provided that this limit exists and is finite. 



(5.1.1) 



Note that the requitement presented by Eq. (5.1.1) means that for any sequence 
{Ax„} which satisfies Ax„ -^ the right-hand side of Eq. (5.1.1) must converge 
to the same number /'(xq) . Thus, we are not restricted for example to positive 
Ax . If by taking two different sequences {Ax^ } , Ax^ -^ and {Ax' } , Ax* — > 
we obtain different limits, the function /(x) does not possess a derivative at Xq . 

The most significant part of Definition 5.1.1 is the word 'exists'. In many cases 
this limit does not exist, and if it does it is not necessarily finite. In terms of 'slope' 
and 'secant', the existence of a derivative means that the secant approaches a line 
called tangent which includes the point (xq,/(xq)) and whose slope is the limit of 
Ay /Ax as Ax ^ (Fig. 5.1.2). While the slope of the particular secant in 
Fig. 5.1.2 is DB/BA, the derivative /'(xq) is given by CBIBA. 




(Xg+Ax,/(Xj|+Ax))^ 



Fig. 5.1.2. The tangent's slope is the derivative / (xg) . 

Example 5.1.1. Consider the function /(x) = x which is defined everywhere. 
For arbitrary Xq we get 
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/(Xo +Ax)- /(xo) _ (xo + Ax) 



Ax 



Ax 



2xo + Ax 



As Ax —^ the right-hand side converges to 2xq which implies /'(xg) = 2xq . 

If a function /(x) possesses a derivative everywhere, i.e. over the function's 
whole domain of definition D ( for example the interval [a,b] in Figs. 5.1.1-2) 
we define the derivative of /(x) as the function 

f'(x) = f'(xo) ,XaeD (5.1.2) 

and say that /(x) dijferentiable everywhere. Thus, (x )' = 2x. The geometrical 



implication is given in Fig. 5.1.3. The original function y = x has the derivative 
function y' = 2x everywhere. At an arbitrary point Xg , the derivative is 2xo and 
equals the slope of the tangent at (xo,Xo ) , i.e. 

y'ixQ) = 2xo = tan(a) (5.1.3) 




Fig. 5.1.3. The function y = x and its derivative y =2x . 



For the sake of simplicity we usually replace the symbol Ax which denotes a 
small increment to x by /z which represents a small number. 

Example 5.1.2. Consider the function y = x" where n is an arbitrary positive 
integer. By Newton's binomial formula we have 



no 
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{x + h) =x + 



^n^ 



vly 



«— 1 1 , 

X 11 + 



X h +...+ 



n-2 



2,,n-2 



X h 



^ n ^ 



n-\ 



xh" + h" 



which implies that 



(X + h) -X n-\ ^ 

- = nx + 



h 



n—2 I , , 

X h + ...+ 



n-\ 






Except for the first term, all the elements at the right-hand side are multiples of 
powers of h and hence converge to zero as h^Q . Consequently, 
)'' = (x")' = «x""' for all X. The case «=0 can be easily included in this 
example. Here the function is 1 and its derivative . If f{x) = c where c is an 
arbitrary constant we also have f'{x) = for all x (the proof is trivial and is left 
for the reader). 

Definition 5.1.2. A function f(x) has a right derivative at Xq if it is defined at a 
right neighborhood of Xg and if the limit 



/'(x + ) = lim 



/(xQ+/i)-/(xo) 
h 



(5.1.4) 



exists and is finite. The notation A ^ 0""" indicates that h is positive and 
approaches zero from the right. A left derivative is similarly defined and by saying 
that f{x) is differentiable over a closed interval [a,i>], we mean that f{x) is 
differentiable inside the interval, has a right derivative at a and a left derivative 
at b . 

A direct result of Definitions 5.1.1 and 5.1.2 is 

Corollary 5.1.1. A function is differentiable at Xg if and only if it has both right 
and left derivatives at Xg and the derivatives are equal. 

The proof is left as an exercise for the reader. 



Example 5.1.3. The function y = 1x1 is differentiable over the interval [0,1] . In 

particular, it has a right derivative at x = : y'{G^ ) = 1 • However, it is not 
differentiable over [-0.1,1] since now it does not possess a derivative at x = 
but rather a right derivative 1 and a left derivative -1 . 

As previously stated, the set of functions that possess derivative is a subset of the 
set C of all the continuous functions. The next theorem confirms this claim. 
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Theorem 5.1.1. A function f{x) which possesses a derivative at Xq is also 
continuous there. 

Proof. 

Given the existence of the limit 

/i^o h 

we obtain 

h->() h->0 n 

which implies the continuity of f(x) at Xq . 

The opposite is not true, i.e. continuity does not imply differentiability. This is 
demonstrated by the following example. 

Example 5.1.4. Let 

[xsin(l/x) , x^O 



fix)- , 

' , x=0 



The continuity of f(x) was previously established for all x . Consider now the 
particular point Xq = . By definition 

fixQ+h)-fixo) ^ hsinjl Ih) ^ 
h h 

Unfortunately, sm{\lh) does not converge as h^>0. For example the choice 
h^ = 2l{nn) , n = 1,2,3,... provides the sequence sin(l//!„) = {1,0,-1,0,1,0,-1,0,...} 
which does not converge in spite of h^ ^>Q . Hence, f{x) is not differentiable at 
X0 = . The function is plotted in Fig. 5.1.4 for 0.01 < x< 0.05 . One can easily 
see that while f{h) converges to zero as h^>0, the slope of f{x) keeps 
vibrating (with increasing speed!) between -1 and 1 . This alone does not prevent 
the existence of derivative at Xq =0 . The derivative there does not exist only 
because sin(l/h) diverges. 
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0.05 




0.05 



-0.05L 
Fig. 5.1.4. A function - continuous but not differentiable at x = . 

In the next example we treat a another vibrating function that due to its faster 
convergence rate at Xq = , is both continuous and differentiable there. 



Example 5.1.5. Consider the function 



f(x) ^ 



X sin(l/x) , x^Q 
, x = 



The continuity of f(x) at Xg = follows from 



lim[/(xo +h)- /(xq)] = limh' sm{l/h) = 

/i->0 /!->0 



since sin(l//z) is bounded and h —> . The differentiability follows using similar 
reasoning, which leads to 

ft^O h h^O 

While the slope (see Fig. 5.1.5) still vibrates in an increasing pace without 
converging (as A — ^ ), the numbers hsin(l/h) converge to zero which 
guarantees the existence of a derivative at Xg = . The non-converging vibration 
of the derivative near Xg = indicates that the derivative is not continuous there. 
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y=x sin{\/x) 




Fig. 5.1.5. A function - continuous and differentiable at x = . 

The use of derivatives in sciences is enormous. A simple example taken from 
mechanics is defining a velocity at a given time. Let the time t be measured along 
the X - axis and let a particle travel along the y - axis. At an arbitrary time t the 

particle is located at y = y(t) . If the particle travels at the time interval /; < f < f j 

its average velocity, between f = fj and f = ^2 is defined as 



Vavitvh) 



yit2)-yih) 



(5.1.5) 



The average velocity does not necessarily reflect on the velocity at all times 

between the specific time limits. If for example, the speed limit on the highway is 

lyiiles Tillies 
60 and v^„(/i,/2) = 50^ the driver may still get a ticket since at certain 



hour 



hour 



times between fj and ^2 h^ might have traveled at 70 



miles 
hour 



while still 



miles 

maintaining an average speed of 50 . However, if fj is fixed and if t2 -t^ 

hour 

decreases and eventually approaches , the average velocity may converge to a 
specific limit called the instantaneous velocity (or simply velocity) at time fj . In 
other words 



v(.,)=lim>^MzZ(^ = /(,^) 



(5.1.6) 
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For example, if a particle's location at time t is ^(f) = f + lOf , its velocity at an 
arbitrary time t is v(f) = 2? + 10 . The proof is left for the student. 

Basic rules for differentiation will be given in the next section. 

PROBLEMS 

1 . Find the derivatives of f(x) = x + x and f(x) = Sx + 5 . 



2. Show that the function 



/W^ 



0,x<0 

^[x ,0<x<l 
l,x>l 



has a derivative except at x = 0,1 . What can be said about these points? 

3. The function f{x) = l/x is not defined at x = . Is it possible to define it at 
this point so that the extended function will possess a derivative 
everywhere? 

4. Use two approaches to show that the function 



fix): 



2 ^ o 

X , X < 1 

4.1, x> 2 



does not have a derivative at x = 2 . Does it possess a left derivative? A right 
derivative? 

5. Consider the function 

I 1 , X rational 
[0 , X irrational 

At what points the function is continuous? Has a derivative? 

6. Consider the function 



fix)- 



0,x<0 

x^ , < X < 1 
2x — 1 , X > 1 



Where does it have a derivative? Explain. 

7. Let /(x) = X , g{x) = X + 2 . Show /'(x) = g'(x) . Any conclusion? 

8. Let fix) be an even differentiable function. Compare between f{a) and 
f\a) for arbitrary a within the domain of /(x) . 

9. Repeat and solve problem 8 if fix) is an odd function. 
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10. Let f(x) = ^x , x>0 . Find where does the function possess a derivative 

and calculate it. 

1 1 . Show that the function 

fix) 

has a derivative everywhere. 

12. Does the function 

fix) 
have a derivative at x = ? Explain 



X , X > 

x^ ,x<0 



X , X > 

X , X <0 



5.2 Basic Properties of Differentiable Functions 

We start with evaluating the derivatives of f{x)± g{x) , f{x)g{x) , f{x)lg(x) 
where /(x) and g{x) are two given differentiable functions defined over the 
same domain. This domain can be an arbitrary open set D within the real axis 
-oo < X < oo although usually, throughout most of the book, we assume, for the 
sake of simplicity, a domain which is a single bounded interval, closed, open or 
semi-open. 

Theorem 5.2.1. For arbitrary differentiable functions /(x) and g{x) 

1. [/(x)±g(x)r=/'(x)±g'(x) 

2. [/(x)g(x)]' = /'(x)g(x) + /(x)g'(x) (5.2.1) 

3. [/(x)/g(x)]' = [/'(x)g(x)-/(x)g'(x)]/[g(x)f , g(x)^0 

Proof. 

1 . For arbitrary h we have 

[/(x + /i) + g(x + /;)]- [/(x) + g(x)] _ /(x + /i)-/(x) ^ g{x + h)-g{x) 
h h h 

and since the right-hand side converges to f'(x) + g'(x) as A ^ this value is 
also the limit of the left-hand side and part 1 of the proof is completed (the proofs 
for sum and difference of functions are identical). 

2. The functions /(x) and g{x) are differentiable and therefore continuous as 
well. By subtracting and adding the same expression we may rewrite 
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f(x + h)g(x + h)-f(x)g(x) ^ f(x + h)g(x + h)- f(x)g(x + h) ^ f(x)g(x + h)-f(x)g(x) 
h h h 

The first term of the right-hand side is g{x + h){[f(x + h)-f{x)]lh). The 
continuity of g{x) guarantees that Van g{x + h) = g{x) and consequently 

h^O h 

Similarly, the second term converges to f{x)g\x) as /i^O which completes 
the proof of part 2. 

3. Let g(x)i^Q at some given x and over a small open interval which includes x 
as well. Then 

f{x + h)l g{x + h)- f{x)l g{x) _ f(x + h)g(x)-f(x)g(x + h) 
h hg(x)g(x + h) 

To complete the proof, the same method of subtracting and adding an identical term, 
should be implemented before letting h ^>0 . It is left as an exercise for the reader. 

Corollary 5.2.1. For arbitrary differentiable function f{x) and constant c we 
have 

[cf(x)]' = cf'(x) (5.2.2) 

The proof is straightforward and is left as an exercise for the reader. 

Example 5.2.1. Let f(x) = x^+x^,g(x) = 5x-l. Then 

f'(x) = 3x +2x , g'(x) = 5 . Consequently 

[f(x)g(x)]' = (3x^ + 2x)(5x - 1) + 5(x^ +x^) = 20x^ + 12x^ - 2x 



fix) 
six) 



(3x^ + 2x)(5x - 1) - 5(x^ +x^) _ 2x(5x^ + x - 1) 
(5x-l)^ (5x-l)^ 



Example 5.2.2. Consider the function /(x) = l/x" where n is a positive integer. 
Then, by using part 3 of Theorem 5.2. 1, we obtain 

, ( 1 ^ 0-x" - Mx""' • 1 _„_l 

,/ (x) = 1 — I = = -nx 



.. , In 

X J X 



This result combined with that of Example 5.1.2 implies 



{x")' = nx"-^ (5.2.3) 



for all integers. 
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We now introduce a new symbol for the derivative which is useful tool in 
obtaining numerous theoretical results. Let y = f(x) denote an arbitrary 

differentiable function. Then dyldx = f'{x) is another symbol, introduced by 
Leibniz, which represents the derivative. Since f'{x) is the limit of the fractions 
Ax I Ay as Ajc — > one can wrongly visualize it as a 'final' quotient where dy and 

dx are 'infinitesimally small' numbers. From a pure mathematical point of view 
this is unacceptable. The derivative is a well defined limit of a sequence and an 
'infinitesimally small' number does not exist. However, the beauty of Leibniz idea 
is first that it helps suspicious students get familiar with the new concept in several 
steps. Also, many theoretical results can be obtained in a most natural by treating 
dyldx as a regular quotient and dx , dy as regular numbers. What important is 

the bottom line and since the final results are correct, these processes are 
acceptable. 

The next result, is very helpful in calculating derivatives of complex functions. 

Theorem 5.2.2. Let y = f{x) ,u = g{t) denote differentiable functions defined 
over open intervals. Consider a point Xq such that y^ = /(xq) is within the 
domain of g{t) , i.e. giy^) exists. Then, the composite function g o f = g{f{x)) 
is differentiable at Xq and 

(g°/)' = g'(3'o)/'(-^o) (5.2.4) 

Proof. 

The proof is quite simple but should be carried with care. Since f(x) is 
continuous, then for sufficiently small Ax , the point /(xg + Ax) = y(j+ Ay is also 
the domain of g{t) . The quantity Ag ={g ° f){xQ + Ax) -{g ° /)(xo) is therefore 
well defined. We can clearly write 

^ = ^^ (5.2.5) 

Ax Ay Ax 

provided that Ay ^0 . There are two cases to be considered: 

Case (a) Assume Ay ^0 for all Ax (already taken small enough!). If Ax ^ 
then Ay^O as well (continuity of /(x)). Since /(x) and g{t) are 
simultaneously differentiable, we get 

Ai^o Ay Ai^o Ax 

Ag , , 

Thus, lim — exists and equals to g (f{xQ))f (xg) as stated. 

Ai^O Ax 
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Case (b) Assume that there is a sequence x„ = Xg + Ax„ where Ajc^ — > such 
that Aji^ = /(x„) -/(xq) =0 . Note that if such sequence does not exist, then 
Ay ^0 for all sufficiently small Ax and we are back to Case 1 . Now, since 
Ay^ =0, we have /'(xq) = 0. Note that the differentiability of f(x) yields that 
using a single specific sequence of x„ = Xg + Ax„ is enough to calculate the 
derivative, provided that Ax^ — > . Thus, g'(/(xo))/'(xQ) = . We still must 
show that Ag/Ax converges to zero as Ax — > . 

Let indeed Ax ^ and consequently Ay— >0. If Ay = then Ag/Ax = 
(since Ag =0). However, if Ay^O, Eq. (5.2.5) holds and Ag/Ay approaches 
g'(yo) while Ay /Ax approaches f'{xQ) = 0. Thus, no matter which route Ay 
takes, we obtain 

\im^ = = g'(f(xo))fXxa) 

Ai^o Ax 

and thus the proof is completed. 

Note that by using Leibniz notation, we just proved 

du du dy 
dx dy dx 

a relation which is trivially correct if du,dy,dx are real numbers and 
dy i^Q , dx^Q . This is a good example for the advantage of this notation in 
performing appropriate shortcuts. 

Example 5.2.3. Let y = f{x) = x +2x and u = g(t) =t +t. The two functions 
are defined and possess derivatives everywhere in R . The composite function is 

gof = (x^+ 2xf + (x^ + 2x) , i.e. gof=x^+6x^+ llx'^ + 8x^ + x^ + 2x . A 
direct calculation provides 

{g o /)' = 6x^ + SOx"^ + 48x^ + 24x^ + 2x + 2 

Using the chain theorem we get 

{g o /)' = (3j;2 + i)(2x + 2) = [3(x2 + 2x)^ + l](2x + 2) 

and the reader can easily verify that the results are identical. 

x^ +1 1 

Example 5.2.4. Consider y = f{x) = , m = g(f) = -- . Let F{x) = g° f . 

x + 1 t 

Then 
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dF ,, , ,,, , 3 2x(x + l)-l-(x^+l) 3(x + l)^(xH2x-l) 

dx / (x + lf (x^+l) 

Example 5.2.5. Let y = x'" " where m,n are arbitrary integers. Then y" = x'" 
and by Theorem 5.2.2 ny"^ y' = mx"^ . Consequently 

m-l 

y = - = — x' 

n y n 

Corollary 5.2.2. If r is rational, the power function y = x'^ yields y' = rx''~ . 
The general case of differentiating a power function is treated next. 

Theorem 5.2.3. For arbitrary real number a , the derivative of y = x" is 

y' = ax"-'. 

The proof of this theorem is beyond the scope of this book. 

Let y = f(x) possess a derivative and consider the inverse function 

y = f^ {x) . Clearly it may not exist and even if it does, it may not possess a 

derivative. For example, the inverse of y = x is not uniquely defined since 

X = ±-yJy . On the other hand the function y = x possesses an inverse x = yy or 

y = yx , should we prefer x, y to keep their usual roles, but the inverse is not 

differentiable at x = . In fact (Vx)' = -(l/3)x" ' ^ c« as x — > and a direct 
computation shows that the derivative does not exist at x = . 

The following theorem yields sufficient conditions for the existence of 
[/" (x)]' and provides its value. 

Theorem 5.2.4. Let y = f(x) , defined over an interval / , denote a differentiable 

strictly monotone function such that f'(x) ^ everywhere. Then, /" (x) 
possess a derivative and 

[f'\x)]' = -^ (5.2.6) 

Note that to use Eq. (5.2.6), we first compute the inverse x = g{y) and then 
substitute g'(y) as the right-hand side of this equation. 
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Proof. 



Let Xq = giyo) and y ^ yo ■ Since f{x) is strictly monotone, so is its inverse, i.e. 
X jt Xf^ as well. Therefore 

8(y)-g(yo) ^ x-Xq ^ 1 

y-yo y-yQ [{y-yQ)i{x-XQ)] 

Being differentiable, f{x) is also continuous and consequently so is x = g(y) . 
Thus, y — > Jo implies x -^ Xg , and since /'(xq) ;^ we get 

8iy)-8iyo) _,,^ 1 



lim °^-" °^-"" = lim 



y^j-o y-yo y^y»[iy-yQ)Kx-Xfi)] /'(xq) 
which completes the proof. 

We will later show that the condition f'{x) ^ already implies that f{x) is 

strictly monotone. 

A relation such as f(x,y) = is called an implicit relation. It includes the set 

of all the pairs (m,v) which satisfies /(m,v) = 0. Every function y = f{x) yields 

the implicit relation g{x,y) = f{x) - y = Q but the opposite is certainly not true. 

2 2 

For example, the relation x - y =0 does not produce a unique function. 
However, if an implicit relation produces a unique function we may not have to 
get the explicit expression for y in order to obtain y' . We simply apply the 
differentiation rule for composite functions. 

Example 5.2.6. Let y = l/x . Clearly, xy -1 = and by using the rule for 

composite functions we get y + xy' = 0, i.e., y' = -y/x = -l/x . Direct 
differentiation produces the same result. The implicit relation 
xy + y + 1 = , y >1 (i.e. the relation is restricted for pairs (m, v) where v > 3 ) is 

less trivial and if a unique function can be defined, then y + 3xy y + y = , i.e., 

/ = -//(l + 3x/). 

Definition 5.2.1. Consider a function f(x) , defined over a domain D . The point 
XqE D is called a local maximum (minimum) point of f(x) , if for some ^ > , 
there exists an interval I = {xq-S,Xq+ S)e D such that /(xo)>/(x) 
(/(xo)</(x))forallxE/. 

Example 5.2.7. The function in Fig. 5.2.1 has a local maximum at Xj and a local 
minimum at X2 . 
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Fig. 5.2.1. Local extrema of fix) . 

An important property of a local extremum (i.e. maximum or minimum), which is 
visually clear from from Fig. 5.2.1 is stated next. 

Theorem 5.2.5. If Xg is a local extremum of a differentiable function f(x) , then 
/'(^o) = 0. 

Proof. 

Let Xq be a local maximum. Then, there exists an open interval 
I = {xq - S,Xq + S) such that /(x)</(xo) for all xe I . Construct (how?) a 
sequence {x„} , « = 1,2,... such that x^ > Xq and lim x„ = Xq . Since 

/(x„) < /(xq) we obtain /'(xq) < . Similarly, by taking a sequence on the left 
of Xq , we get /'(xq) > . Hence /'(xq) = . 

A direct result of Theorem 5.2.5 is the famous Rolle's theorem. 



Theorem 5.2.6 (Rolle's theorem). Let f(x) be a differentiable function over a 
closed interval [a,b] , and let f{a) = f{b) = . Then, there exists Xq : a < Xq < Z? 
such that /'(xq) = . 

Proof. 

If fix) = everywhere then /'(x) = for all X and the theorem's claim is 
trivial. If the function is not identically zero, assume for example that it attains 
positive values. Since /(x) is differentiable, it is continuous as well and 



122 5 Differentiable Functions 

possesses a global maximum at some Xq within {a,b) (why?). This, by 
Theorem 5.2.5, implies /'(xo) = 0. If f{x) attains negative values, it must 
possess a global minimum at some Xq : a < Xq < b and again /'(xg) = . 

Note that Xq is not unique. If for example f(x) = sin(x) , a = , b = Itt , then 
atxQ =.^/2 , Xq =3;r/2 , the function attains local maximum and minimum 
respectively and its derivative there is zero. 

The validity of Rolle's theorem can be extended to a more general case. 

Corollary 5.2.3. If f{x) of Theorem 5.2.6 satisfies only the equality 

f{a) = f(b) there still exists Xq : a < Xq < Z? such that /'(xg) = . 

The reader will find the proof straightforward by applying Rolle's theorem to the 
function g(x) = f(x) - f(a) which satisfy g{a) = g(b) = and g'(x) = f'{x) . 

The next result which follows almost directly from Rolle's theorem, is a powerful 
tool in the development of calculus - particularly the presentation of a function as 
a power series. 

Theorem 5.2.7 (First mean- value theorem). Let /(x) denote a differentiable 
function over a closed interval {a,b'\ . Then there exists ^ : a < ^ <b such that 

f(b)-f(a) = f'(^)(b-a) (5.2.7) 

Note that ^ , as in Rolle's theorem is not necessarily unique. 

Proof. 

Define 

g(x) = /(x)-/^^^-^(x-a) 
b — a 

The function g(x) is differentiable over [a,b] just like /(x) (why?) and since it 
yields g{a) = g{b) = f{a) we can implement Corollary 5.2.3 and conclude the 
existence of g for which g'{^) = . Consequently 

b — a 
which implies Eq. (5.2.7). 

The mean-value theorem actually states that the average change of a function 
between two points a and b , although usually different from the derivatives at 
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these points, must be equal to the function's slope at some middle point. The idea 
presented at the proof of Theorem 5.2.7 is to rotate the function f{x) until its 
values at the endpoints are the same (Fig. 5.2.2). 




a I, b 

Fig. 5.2.2. Getting the mean- value theorem by rotating f{x) . 

The next result is a straightforward conclusion from Theorem 5.2.7. 

Theorem 5.2.8 (Second mean-value theorem). Let f{x) and g{x) be 
differentiable over the interval {a,b'\ such that g(b) ^ g(a) and g\x) i^ 
everywhere. Then, there exists an interim point ^ :a < ^ <b for which 

A^) _ f{b)-f(a) 
g\^) g(b)-g{a) 

Proof. 

Define 

F{x) = f(x)[g(b) - g(a)] - g(x)[f(b) - f(a)] (5.2.9) 

which implies 

F(a) = F(b) = f(a)g(b)-g(a)f(b) 

and consequently, by the first mean-value theorem, there exists a 
^ :a <^ <b such that F'{^) = . The rest is straightforward. 
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While the first mean-value theorem, by virtue of Fig. 5.2.2, seems visually 
logical, a geometric interpretation of Theorem 5.2.8 is not so clear since we have to 
observe the graph of F{x) (Eq. (5.2.9)) rather than those given of f(x) and g{x) . 

We close this section by showing the relation between a function's derivative and 
its domains of increasing and decreasing. 

Theorem 5.2.9. Let y = f(x) possess a continuous derivative over an arbitrary 
domain D and let /'(xq) >0. Then f{x) is strictly increasing at Xq , i.e. at a 
small interval which contains Xq . 

Proof. 

The continuity of f'{x) guarantees the existence of a small interval / around Xq 
such that f'(x)>0,xel. Let Xi,X2 denote arbitrary points at / such that 
Xj > X[ . Then, by the first mean value theorem we have, there exists an interim 
point ^ : Xj < ^ < X2 such that 

/(x2)-/(xi) = (x2-Xi)/'(^)>0 

i.e., /(xj) < /(X2) . Similarly, if /'(xg) < then /(x) is strictly decreasing at Xg . 

Example 5.2.8. The function y = x - x yields y' = 4x - 2x . Simple algebra 
shows that the function is strictly increasing at the semi infinite interval 
(V2/2,oo) and at (-v2/2,0) and is strictly decreasing at the semi infinite 

interval (-oo,-V2/2) and at (0,V2/2) . 
PROBLEMS 

1. Find the derivatives of (a) /(x) = x^ -x^ +x + l (b) /(x) = lOOx'" + (V^)^ 
(c)/(x) = x"^V^ (d) /(x) = (x^+x)/(x^-3Vx). 

2. Find the derivatives of (a) /(x) = (x^ + x)^ (b) /(x) = [(x + V^)/(x^ + 1/x)]^ 

3. For the following functions find the inverses and their derivatives: 
(a)/(x) = 2x^ (b) /(x) = l/(V^ + l). 

4. Show that the inverse of /(x) = x + x exists and find its derivative. 

5. Assume that a differentiable function y satisfies the equation x y + yx = 
and calculate y' . As previously stated this implicit relation by itself does not 
guarantee that a unique function y = f(x) exists. 

6. Let f(x,y) = y x + x y -2 = and assume that y{x) is a differentiable 
function which solves this equation, (a) Calculate y' . (b) Show /(1,1) = 0. 
Does that mean ^(1) = 1 ? (c) Assume ^(1) = 1 and calculate y'(V) 
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7. Show that the derivative of /(x) = X +x attains the value 10 within [1,2] . 

8. (a) Show that for some Xq within [1,2], the functions f(x) = x +x and 
g(x) = x^ +x satisfy /'(xo)/g'(xo) = 2.5 . 

(b) Calculate Xq . 

9. Let f(x) = 3x , g{x) = x over the interval [2,3] . Find the unique point i^ 
inside the interval, at which the second mean- value theorem holds for f{x) 
and g{x) . 

10. How can you simplify the proof of Theorem 5.2.8 in the case f(a) = f(b) ? 

5.3 Derivatives of Special Functions 

We are now ready to calculate the derivatives of the trigonometric and logarithmic 
functions and their inverses. Like most traditional authors we start this section 
with the function y = sin(x) . 

Theorem 5.3.1. Let x denote an arbitrary angle measured in radians. Then 

sin(x) 



lim- 



I 



(5.3.1) 



Proof. 



Consider a sector of the unit circle with arc (angle) x (Fig. 5.3.1) measured in 
radians. A basic statement from elementary plane geometry is that the area S of this 
sector is its arc multiplied by half the radius, i.e. S = {x-l)/2 . This area is bounded 
between the triangle OAB with area ^i (from below) and the triangle OAC with 
area ^j (from above). 




tan(x) 



D A 

Fig. 5.3.1. Obtaining the inequality of Eq. (5.3.2) 
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Hence 

AOBD I- sin(x) „ _l-x OAAC I- tan(x) 

O] — — 'Co — < O-l — — 

2 2 2 2 2 

which leads to (how?) 

1 < — ^ < — *— (5.3.2) 

sin(x) cos(x) 

The first part of this inequality, i.e., < sin(x) < x clearly guarantees 
limsin(x) = . By virtue of the identity sin (x) + cos (x) = 1 , we therefore also 

have limcos(x) = 1 . Thus, the whole inequality of Eq. (5.3.2) yields 

X 

lim = 1 which completes the proof. 

x^o sin(x) 

The next result is a direct result of Theorem 5.3.1. 
Theorem 5.3.2. (sin(x))' = cos(x) . 

Proof. 

Using a basic trigonometric identity we obtain 

sin(x + /i) — sin(x) 2sin(/z/2)cos(x + /z/2) sm(h/2) , , ,^, 

= — = — cos(x + hL) 

h h (h/2) ' 

By applying Theorem 5.3.1 and basic results from the theory of limits we 
therefore have 

,. sin(x + /i) - sin(x) ,. sin(h/2) ,. , , ,^, 

hm — ^^ ^^ = hm — ^-^-^limcos(x + hi) = cos(x) 

ft^o h h^o {h/2) h^o 

Corollary 5.3.1. (cos(x))' = -sin(x) . 
Proof. 

Rewrite cos(x) = sin(.?r/2 - x) . If we define z = 7r/2 - x then by virtue of 
Theorem 5.2.2 we get 

(cos(x))' = (sm(z))[ z'^ = cos(z) • (-1) = -cos(;r/2 -x)=- sin(x) 
The notation z^ denotes differentiation of the function z by the variable X . 

Example 5.3.1. Let f(x) = tan(x) . Then 
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, ^^, I sin(x) I cos(x) • cos(x) - (-sin(x) • sin(j«:) 1 
(tan(x)) = — ^ = ^^ ^-^ ^^ ^ = 

l^COS(x)j COS (x) COS (x) 

and similarly (cot(x))' = -l/sin (x) . 

Next we will differentiate the logarithmic function with base e (see 
Example 3.7.2), called the natural logarithmic function and denoted by ln(x) . 

Theorem 5.3.3. The function y = \n(x) , x > yields y' = 1/x . 

Proof. 

For arbitrary x > we have 

\n(x + h)-ln(x) _l(x\ f x + h ) 1 f x + h V''' _ 1 f x^"^'' 



h x\h) \ h ) X \ h 

Since xlh — > oo as A — > we get 

,. (^ xY^/" 

hm 1 + — =e 
h-^\ h) 

and the continuity of ln(x) at all x , particularly at x = e , yields 

\n{x + h)-\n{x) _ 1 

h^Q h X 

Corollary 5.3.2. The function y = e^ yields y' = y ■ 

Proof. 

The inverse function of the exponential function is x = \\\{y) . Using the 
differentiation rule for a composite function we get 

Hence y' = y ■ 

Example 5.3.2. Consider the function y = xln[sin(x)] . Its domain of definition is, 
sin(x) > i.e., Ikn < x< (2k + \)7i for arbitrary integer k . The derivative (again 
we must use Theorem 5.2.2) 

y = ln[sin(x)] + xcot(x) 

is defined over the same domain. However, finding the zeroes of y,y' or y" is 
certainly not an easy task. 
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The inverse functions of tlie trigonometric and logarithmic functions are just as 
important and used in numerous appHcations. We will define them and calculate 
their derivatives using, as we did for the exponential function (Corollary 5.3.2) the 
rules for composite functions. 

Definition 5.3.1. The inverse function of y = sin(x) is defined as 

x=g(3'),-l<3'<l where for each y the value x is taken so that y = sin(x) . 
Since sin(x) is periodic x is not uniquely defined. We usually choose the main 
branch of the inverse function for which -7r/2 < x< njl and after switching 
back the roles of x and y , write y = arcsin(x) or y = sin"'(x) where -1 < .x: < 1 

and -7r/2 <y<7r/2 (Fig. 5.3.2). 




(1,71) 



Fig. 5.3.2. The function y = sin (x) - main branch. 



The inverse of y = cos(x) is also not uniquely defined and to overcome this 
obstacle we define y = cos" (x) over -1 < x < 1 such that for a given x the value 
of cos" (x) is the number y ■.0< y < 71 for which x = cos(y) . 



Definition 5.3.2. The inverse function of y = tan(x) (main branch) is 
y = arctan(x) or y = tan" (x) (Fig. 5.3.3) where for each x : -oo < x < oo , the 
value y is the unique number within the interval [-7r/2,7r/2] for which 
X = tan{y) . 
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Fig. 5.3.3. The function y = tan (x) - main branch. 

The inverse of y = cot(x) is similarly defined except that for each 
X : -oo < X < oo , we choose the unique y :0< y < tt which satisfies x = cot(y) . 




Fig. 5.3.4. The logarithmic and exponential functions. 
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The inverse of j = ln(;c) is clearly y =e'^ , defined over the whole x - axis and 
attains only positive values (Fig. 5.3.4). 

The best way to calculate derivatives of inverse functions is by applying 
Theorem 5.2.4 since we do not actually have to invert the function before 
calculating the inverse's derivative. 

Example 5.3.3. The function y = sin" (x) yields y' = l/CsinC};))' = Xjco'&iy) . The 
relation x = sin(3') yields 



±4\-x- 



but being restricted to the main branch we get y' = \j 4^ — x . 

Example 5.3.4. The function y = tan" (x) yields y = l/(tan(}'))' = cos (y) . By 

2 / 2 

virtue of x = tan(}') and the identity 1 + tan {y) = 1/cos (y) we conclude 



1 + x^ 



PROBLEMS 

1 . Differentiate the identity sin (x) + cos (x) = 1 and show that both sides 
provide . 

2. Write a detailed proof to the claim limcos(x + h) = cos(x) . 

3. Find y' from the implicit relation x e-'+ln(2 + x +y ) = 0. Does this 
exercise make sense? 

4. Show that the function f(x) = x sin(l/x) , defined as at x = , has a 
continuous derivative everywhere. 

5. Find the domain of definition and derivative of the functions: 

(a) f(x) = ln(e^sm(x)) (b) f (x) = Un-\ln(x)) (c) /(x) = e^'"('"<^» . 

6. Find the derivative of y = x^ (Hint: apply the logarithmic function on 
both sides). 

7. Find domains of definition and derivatives for: 

(a)/(x) = x'"W (b) /(x) = [ln(x)]'"<^' (c) tan-'(e'=) . 

8. The hyperbolic cosine and sine functions are defined as 

X , -X 

€ -\- 6 
COSh(x) = , - oo < X < oo 
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e — e 
sinh(x) = , - oo < X < oo 

2 

Show: (a) cosh^(x)-sinh^(x) = 1 (b) cosh'(x) = sinh(x) (c) sinh'(x) = cosh(x) . 
9. Calculate the derivatives of tanh(x) = sinh(x)/cosh(x) , coth(x) = cosh(x)/sinh(x) . 

10. Find the extremum points of cosh(x) , sinh(x) , draw the functions and 
compare with your results. 

11. Find the domains where the following functions are defined: (a) ln[cosh(x)] 

(b) ^tanh(x) - cosh(x) (c) ln[cos(x) -0.5cosh(x)] . 

12. Calculate the derivatives of the following functions: (a) x^' (b) {x^Y 

5.4 Higher Order Derivatives; Taylor's Theorem 

In this section we introduce higher order derivatives, obtain Taylor's theorem - 
one of the most useful tools in analysis, and present rules for studying the 
behavior of an arbitrary differentiable function. 

Definition 5.4.1. Let f{x) denote a differentiable function defined over an 
interval [a,b'\ and let Xq be an arbitrary point within the interval. The second 
derivative of /(x) at x = Xq is the number 

/'(xo) = lim^^^^^^^4^^^ (5.4.1) 

provided that the limit at the right-hand side exists. If the limit exists for all 
XQe[a,ii] then /(x) possess a second derivative /'(x) (written also as /"^ ^(x) 

or as — y) over [a,b] (The derivatives at the endpoints are right derivative at a 
dx 

and left derivative at fo ). 

Example 5.4.1. The function y = x + sin(x) satisfies y' = 3x + cos(x) and its 
second derivative exists everywhere and equals to y" = (ix- sin(x) . 

Note that in order for /'(xq) to exist, the first derivative /'(x) must exist at least 
in a small neighborhood ofxg, otherwise the number /"(xg) is simply not 
defined. However, this is only a necessary condition since the expressions at the 
right-hand side of Eq. (5.4.1) must not only exist but also converge to a finite 
number. 

Example 5.4.2. The function 
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^^ ^ fx^ sin(l/x) , X ;^ 
fix) = \ 

is differentiable everywhere and satisfies /'(O) = as can be easily seen. We also 
have /'(x) = 3x sin(l/x)-xcos(l/x) for all x ;^ . However, the expression 

f'(h) - /'(O) _ 3/z^ sin(l//!) - hcos(l/h) 



h 



■-3hsm(l/h)-cos(l/h) 



although bounded, does not converge but rather oscillates as h -^0 . 

Consequently, the second derivative does not exist at x = . 

We previously defined the instantaneous velocity of an object as the derivative 

dx 
x'(t) = — where x(t) denotes the location of the object at time / . Consider the 
dt 

velocities x'(/[ ) , x'(/2 ) at times f^fj respectively. The quantity 

««.(^.^2) = ^^^^^^^^ (5.4.2) 

is defined as the average acceleration of the object between fj and ^2 . If this 
quantity converges as ^2 approaches /] , the limit is called the instantaneous 

d^x 
acceleration of the object at time t^ and denoted by a{t{) = x"{t^) = — ^(?i) . If a 

dt 

force F is acting on an object with mass m and acceleration a , then by Newton's 

second law of motion 

d^x 
F = ma = m — — (5.4.3) 

dt^ 

The next definition completes the definition of higher order derivatives. 

Definition 5.4.2. Let the n-th (n>2) derivative of /(x) exist in a small 
neighborhood of Xq . If the limit 

^^_^^ f'"'(^o+h)-f'"\x,) 

h^O h 

exists, it is called the {n + \)-th derivative of /(x) at Xq and denoted by 

/(-'>(xo)(orby4^). 
dx 

If /'(x),/''(x),...,/*"^(x) exist and are continuous over the interval [a,b] , we 
write /(x)e C"[a,ii] (the continuity of /^ ^(x),1<^<m-1 is guaranteed by 
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the existence of f "'(x) - why?), i.e. f{x) belongs to the set of all functions 
which possess n continuous derivatives over the closed interval [a,b]. We 

similarly define the function sets C" (a, Z?) , C"[a,b) andC"(a,b]. 

Example 5.4.3. The function f(x) = l/cos(x) satisfies f'(x) = sin(x)/cos (x) 
and /"(x) = l/cos(x) + 2sin (x)/cos (x) everywhere except at the discrete 
points x^=(n + l/2);r where n is an arbitrary integer. Such a point is called 
singular and since cos(x„ ) = , the function and all its derivatives are not defined 
at these points (Fig. 5.4.1). 




i-n/lfl) (Jt/2,0) 





Fig. 5.4.1. Domain of defmition of f{x) = l/cos(x) : -oo <x<oo,x^{n + l/2)7r 

The next result is an extension to Theorem 5.2.7 and one of the most significant 
results in calculus. It enables us to approximate a function, occasionally quite 
complex, by a polynomial of arbitrary degree, provided that several derivatives of 
the function exist and can be calculated at a single point. 

Theorem 5.4.1 (Taylor's theorem). Let f{x) possess m + 1 derivatives in some 
neighborhood of x = Xq. Then 

f(x) = f(xa) + fXxa)(x-Xo) + ^-^(x-Xaf+...+ ^ f°\ x-Xo)"+J?„ 

2! m 

where /?„ , called the remainder, is given by 
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Rn = \ \^J {x-x,r' (5.4.4) 

(m + 1)! 

where ^ is some interim point between Xq and x . 

Thus if R^ is known to be relatively small, the function f(x) can be 
approximated by Taylor's polynomial 

P„(x) = f(Xo) + f(Xo)(x-XQ) + (x-Xq) +... + ; (x-Xq) 

2! n\ 

A polynomial approximation has clear advantages as its properties are known and 
it is also easily manipulated. 

Proof. 

Let b denote an arbitrary fixed point in the neighborhood of Xq . In order to show 
that Taylor's formula holds for x = b , define 

Fix) = f(x) + f'{x){b -x) + ...+ ^ " '-'^^ {b - x)" + -^^^ " ■^^" 



where A is a constant yet to be determined and x varies between Xq and b . 
Clearly, F{x) is differentiable between Xg and Z? , and F(/7) = /(Z7) . However, 
we may force F{xq) = f{b) as well, by a proper choice of A , namely, 

A = F(xo)-/(xo)-/'(xo)(Z7-Xo)-...-^-^(Z7-Xo)" (5.4.5) 

Since F(xo) = F{b) = f{b) , Rolle's theorem can be applied to obtain F\^) = 
for some interim point ^ between Xq and b . Simple algebra provides 

/<"+'>(x)^^ ^„ A{n + \)(b-xT 



F'(x) = ^ ^-^{b-x) 



(b-xo)"^^ 



and by virtue of F\^) = we get that A , already determined by Eq. (5.4.5) must 
also satisfy 



(m + 1)! 



-{b-x^Y 



The remaining of the proof, i.e., substituting in Eq. (5.4.5) and replacing F{x^■^) 
by f{b) is straightforward and is left for the reader. 

Note that by applying Taylor's theorem for m = we obtain the familiar first 
mean-value theorem (Theorem 5.2.6). 
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Although Taylor's theorem is usually applied to approximate functions, we will 
first show how it is used to obtain simple rules for determining special points 
within a function's domain - the stationary points. 

Definition 5.4.3. A point jcq where /'(xq) = is called a stationary point. 

Definition 5.4.4. A function /(x) is called concave up in an interval I = {a,b) , 
if f'(x) is strictly increasing in / and concave down if f'(x) is strictly 
decreasing in / . 

Definition 5.4.5. Let f{x) denote a differentiable function over an interval / . A 
point XqG I is called an inflection point if f{x) is concave up at the left (right) 
of Xy and concave down at the right (left) of Xq . 

Example 5.4.4. The stationary points of /(x) = X -3x +x are determined by 
/'(x) = 4x - 9x + 2x = , and are consequently Xj = , Xj = 8 , Xj = 0.25 . 

Example 5.4.5. Let /(x) = x ,-oo<jc<<x'. The first derivative /'(x) = 3x is 
clearly strictly increasing for x > which implies that /(x) is concave up there. 
For X < , /(x) is concave down (why?). 

We usually distinguish between two classes of stationary points: (a) extremum 
points; and (b) other points which are usually inflection points if the function is 
twice continuously differentiable. The next result concerns the first class. 

Theorem 5.4.2. Consider a function /(x)e C ia,b) such that /'(xo) = and 
/"(xq) itQ _ Then Xq is a local maximum of /(x) if /''(xq)<0 and a local 
minimum of /(x) if /'(xq) > . 

Proof. 

By Taylor's theorem, we get for arbitrary x e (a,b) 



/(x)=/(xo)+(x-xo)r(xo)+ ^" ^^^ n^) 



(X Xg) ^,^ 



where ^ (which depends on x) is between Xq and x. Since /'(xo) = this 
yields 

/(-^)=/(-^o)+^^^r(a 
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The continuity of f"{x) guarantees that /''(xo)and f"{^) have the same sign 
provided that x is sufficiently close to Xq . Therefore, the sign of the remainder 



R. 



(x-Xq) 

2! 



/'(^ 



is the same on both sides of Xq . The rest of the proof is straightforward and is left 
for the reader. 

Example 5.4.5. Define y = x^ - 2x^ + xover the interval [0,2] (Fig. 5.4.2). The 
derivative y' = 3x - 4x + 1 vanishes at Xj = 1/3 and ^2 = 1 . The second derivative 
y" = 6x-4 satisfies y"(l) = 2 > and y"{l/3) = -2 < , implying that Xj is a 
local maximum and X2 is a local minimum of y . 




3,2, 
y=x —2x +x 



3 2 

Fig. 5.4.2. Graph and extremum points of y = x —2x + x . 



2 - 

: X e 



over the interval [-1,3] 



2 \ -X 



and 



Example 5.4.6. Consider the function y ■ 

(Fig. 5.4.3). The first and second derivatives are y' = (2x- x^)e 
y ={2-Ax + x )e respectively. The solutions of y =0 are Xj = , X2 = 2 
which are local minimum and maximum respectively, since y''(Q) = 2/e > and 
y^l) = -21 e^ < . 
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Fig. 5.4.3. Graph and extremum points of y = x e '^ . 

Clearly, the requirements of Theorem 5.4.2 present sufficient conditions for the 
existence of extremum points. They are certainly not necessary conditions. For 
example, the function y = \x\ is differentiable everywhere except at x = where 
it attains a minimum. Another function defined as 



/W^ 



0,x^3,5 
-l,x = 3 

7,x = 5 



has a minimum at x = 3 , maximum at x = 5 and is not differentiable or even 
continuous at these points. 

By virtue of Theorem 5.4.2, it is clear that in the case of /'(xg) = /"(xq) = , 
one cannot, without further investigation, determine the nature of the point. 
Indeed, if /'(xq) = , Xq is usually either a local maximum, a local minimum, or 

an inflection point where the function moves from a concave up (down) zone to a 
concave down (up) zone. 

The next result is an extension of Theorem 5.4.2 which enables to carry on the 
search for extremum points in the case of /'(xg) = /"(xg) = . We first prove the 
following lemma. 



Lemma 5.4.1. If f{x)eC'[a,b] and f\x)>Q (f"(x)<0) , then /(x) is 
concave up (concave down) over [a,b] . 
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Proof. 

Let f"(x) > . If Xj < X2 , then, by applying the first mean value theorem to f'{x) 
we get 

f'(^2 ) - f'i^l ) = (-^2 - -^1 )/'(^) 

for some interim ^ : a < x^ < X2 < b . Thus f'{x) is strictly increasing, i.e. f{x) is 
concave up. The other case is similarly treated. 

Theorem 5.4.3. Let f(x) e C"[a,b] ,n>2 and let 

f'(xa) = f\xo) = ... = f""\xo) = , /<"'^'>(xo)^0 , m<n (5.4.6) 
Then, Xq is: 

local maximum, if: m odd and / '"'^ '{xq)<0 
local minimum, if: m odd and / '"'^ '(xq)>0 
inflection point, if : m even and f '"'''' (xq) it 

Proof. 

By Taylor's theorem we get (for x close to Xq ) 

/(x) = /(xo)+ ^^/"^ /<'"^'>(a (5.4.7) 

where ^ is an interim point between Xq and x .If m is odd the remainder's sign 

is determined by the sign of / (-'^o) '^^ ^"^^ sides of Xq . If it is positive we 
have f{x) > /(xq) and a local minimum is obtained. If it is negative, we get a 

local maximum. If m is even (m is at least 2 ), we apply Taylor's theorem to the 
second derivative f"(x) and obtain 

/ \m-l 

r(.^)= ^"~"i„ f""'''(.t) (5.4.8) 

(m-iy. 

where ^ is some interim point between Xq and x (usually ^ '^^)- If 
/ '" {^Xq ) > , then /'(x) > at the right of Xq provided that x is 
sufficiently close to Xg . This implies that /(x) is concave up for x>Xo. 
Similarly, /(x) is concave down forx at the left of Xq and sufficiently close to 

it. The case / "^ (x0)<O provides a concave up zone at the left of Xg and a 
concave down zone at the right. 
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Example 5.4.7. Let f (x) = x\n(x) - 2x . The function domain of definition is 
X itQ _ The stationary points are the solutions of f'{x) = ln(x) -1 = 0. We have a 
single solution Xq = e at which /"(xq) = 1/xq = 1/e > , which implies that Xq is 
a local minimum. 

Example 5.4.8. Let y = x + sin(x) . The stationary points are obtained by 
y' = 1 + cos(jc) = which implies x = X/^ = {2k + l);r for arbitrary integer k . 
Since the second derivative 3'' = -sin(ji:) yields ^''(xj. ) = , we must calculate the 
third derivative in order to determine the nature of the stationary points. Since 
3'" = -cos(.t) we get y"'{xi^) = \ and consequently all x^;. are an inflection points 
(Fig. 5.4.4). 




Fig. 5.4.4. Stationary points of y = x + sin(x) : only inflection points. 



Note that an inflection point of a function f{x) is not necessarily a stationary 
point. Consider a function f(x)e C [a,b] for which Xq is an inflection point 
such that /'(xq) i^O . Assume that the function is concave up to the right and 
concave down to the left of this point.. If in addition f"(x) > at the right and 
f"{x) < at the left of Xq (these are sufficient conditions for the concave up and 
down zones respectively) we must have /"(xq) = . If now /"(xq) i^ , then Xg 
is simultaneously an inflection point of f(x) and an extremum point of f'{x) . 
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Example 5.4.9. The function y = 1x — x has three extremum points and two 
inflection points that are not stationary (Fig. 5.4.5). Indeed, y' = 'i,x -2x = 
yields two local minima at Xj 2 = ±0.5 and a single maximum at X3 = . The 

solutions of y" = 24 jc -2 = 0, i.e. X4 5 = ±l/2v3 , are inflection points that are 
not stationary points of y but rather extremum points of y . Indeed, the second 
derivative of y' which is y" = A%x provides y'"{Xi)>Q (minimum) and 
y"'{x^') < (maximum). 



-1/2^3 1/^V3 0.5 




4 ? 
Fig. 5.4.5. Extremum and inflection points foY y = 2x — x . 

PROBLEMS 

1. Find the second derivatives of the following functions: 

(a) >' = sin(e-') (b) y = e'' (c) y = ln[sin(x) + tan(x)] (d) >> = e'/(x + e-*) 

2. Find the stationary points of f{x) = x"e^^ ,-<^<x<oo and their types. 

3. Find domain of definition, derivative, stationary points and their types and 
domains of monotonic increase and decrease for the following functions: 

(a) y = X -X +x (b) y = x\n([ + x) (c) >' = sin(x) + cos(x) 

4. Let f{x) = 2x -1/x . Find the domain of definition, stationary points and 

their types, domains of monotonic increase and decrease and draw the 
function. 

5. Find domain of definition and stationary points for f{x) = asin(;<:) +bcos(x) . 

Assume a^ +b^ >0 . 

6. Show that the function f(x) = x'"(a-x)",0<x<a , where m,n are 
positive integers, has a global maximum inside the interval and find it. 
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7. Find the stationary points of /(x) = X -2x +x and their types. 

8. Use the first mean- value theorem to show e'^ >l + x for arbitrary x>0. 
Find the stationary points of f{x) = e^ -I- x and their types. 

9. Let f{x) = sin(x) , Xq = . Use Taylor's theorem to evaluate R^ and bound 
it for W ^ 1 . 

10. Let f(x) = xe^ , Xq = . Use Taylor's theorem to evaluate R^ . 

5.5 L'Hospital's Rules 

Quite often it is necessary to calculate limits such as 

lim/W 

x^a g{x) 

where a is usually an endpoint of an interval where the functions are defined, and 
either lim/(x) = limg(x) = or lim/(x) = limg(x) = 00 . In either case the 

X— >a X— >fl X— >a X— >a 

substitution x = a leads to nowhere. The technique used to obtain these limits - if 
they exist is based on the first mean-value theorem and is known as L'Hospital's 
rules which help treat expressions such as 0/0 and 00/00 . 

Theorem 5.5.1 (L'Hospital's first rule). Let /(x),g(x) be continuous over a 

finite interval \a,b\, differentiable over (a.h) such that 

lim /(x)= lim g(x) = and let g'(x)^Oin a right neighborhood of a (not 

X— >a^ X— >a^ 

including a itself). Then 

(1) If lim ^^ = A (finite) then lim ^^ = A . 

x^a* g (x) x^a* g(x) 

(2) If lim ^^^ = 00 then lim ^^ = 00. 

x->a+ g (x) x^a* g{x) 

(3) If lim ^^^ = -00 then lim ^^ = -00. 

x->a+ g (x) x^a* g{x) 

Proof. 

Case (1) By virtue of Theorem 5.2.7 we have 

fix) _ f(x)-f(a) _ f'{^) 
g(x) g(x)-g(a) g'(^) 

for some interim point ^:a<^<x<b provided that x is in the neighborhood 
where g'(x) ^0. Since the right-hand side converges to A as x^a"*", so does 
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the left-hand side. Obviously, the same argument basically holds for Cases (2) and 
(3) as well. The exact details which include the separate definition of converging 
to oo , are left as an exercise for the reader 

It should be noted that in Theorems 5.5.1-5.5.2 we can replace a with b and 

a""" with b~ since the limits are obtained by approaching b from the left. 

Example 5.5.1. Let f(x) = x^-l,g(x) = x^+x-2,a=l. Here 
/(I) = g(l) = and by L'Hospital's first rule 



-r->l g(x) x^l 2x + \ 

Example 5.5.2. Let/(x) = sin(x) , g{x) = \- cos(x) , a = . All the requirements 
of Theorem 5.5.1 are satisfied and consequently 

sin(x) ,. cos(x) 
lim = lim = oo 

x^o 1 - cos(x) t^o sin(x) 

If lim [/'(x)/g'(x)] is also of the form 0/0 or oo/oowe may check whether 

/' , g' satisfy the requirements of L'Hospital's first rule. If so we can continue 
and try to obtain lim [f"(x)l g'Xx)} etc. 

Example 5.5.3. Let f{x) = x^ - 4x^ , g(x) = x^ + x'* ,a = . Here 

,. x^-4x^ ,. 3x^-8x ,. 6x-8 
lim — — = lim = lim 



x^o+ x^ + x'* x^o* 2x + 4x^ .1^0+ 2 + 1 2x^ 

In many applications both numerator and denominator approach infinity as 
X — » fl""" . This problem is treated next. 

Theorem 5.5.2 (L'Hospital's second rule). Let f(x),g(x) be differentiable over 
(a,b) such that lim/(x)= lim g{x) = o° and let g'{x)^Oin a right 

x^>a'^ x^a' 

neighborhood of a (not including a itself). Then 

(1) If lim ^^ = A (finite) then lim ^^ = A . 

x^a* g (x) x^a* g{x) 

(2) If lim ^^^ = 00 then lim ^^ = 00. 

x->a+ g (x) x^a* g{x) 

(3) If lim ^^^ = -00 then lim ^^ = -00. 

x->a+ g (x) x^a* g{x) 
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Proof. 

The proof is more complex than that of the previous result. We will give it in 
detail only for Case 1 . Given an arbitrary £ >0 one needs to show the existence 
of <5 > such that if a < x <a + S then \f(x)/g(x) - A\< e . However, by virtue 
of lim [f'{x)/g'(x)] = A , given e^ = e/3 there exists Jj > such that 



fix) 



Xx) 



< £i , a < X < a + Si 



(5.5.1) 



By applying Theorem 5.2.7 we get 

f(x)-f(a + SO _ f'(^) 
g(x)-g(a + SO g'(^) 



a < x< a + Si 



where g is an interim point between a and a + Si, assuming that ^j is 
sufficiently close to a so that both denominators of the last equality are not zero 
and g(x) ^ as well. Hence 



fix) fia + SQ 
gix) gjx) _ f'i^) 

8ia + ^i) g'i^) 

gix) 



a < X <a + Si 



1- 



which leads to 



fix) 
six) 



^ gia + SOy'i^) ^ fia + SQ 



gix) J g'i^) gix) 



(5.5.2) 



By virtue of lim gix) = oo , given an arbitrary £"2 (i^ot yet determined!) we can 



find S2 '■ a < S2 < Si such that 



gia + Si) 



gix) 



<e. 



fia + Si) 



gix) 



<e. 



a < X < So 



(5.5.3) 



Therefore if we rewrite Eq. (5.5.2) as 



fix) ^^mi_^_gia^Si)riil^fia^Si) ^^^^^ 



gix) g'i^) gix) g'i^) 

we get, using Eqs. (5.5.2-3) the following relation: 



gix) 



fix) 
gix) 

and finally we obtain 



g'i^) 



+ ^2 (^ + £"i ) + £"2 ' a < X < ^2 
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fix) 



gix) 



<Si+e2[l + A + e^] , a<x<S2 



We now choose £2 = £"i /(I + A + £[ ) and get 



fix) 



gix) 



<2£i 



le 



<e ,a< x<S 



for the choice d = 82- This completes the proof. 

The proof in Cases (2) and (3) are left as challenging exercises for the 
interested reader. 

The results presented in Theorems 5.5.1-5.5.2 can be easily extended to include 
the case a = °o . 

Theorem 5.5.3. The results in Theorems 5.5.1-5.5.2 hold for a = -=0 . 
Proof. 

In order to show validity of the previous results for a = 00 , it is sufficient to show 
that each case with a = 00 is equivalent to a case with a finite a - already covered 
by Theorems 5.5.1-5.5.2. 

Let f{x),g{x) be differentiable over (Z7,oo) such that g'{x)^0 for 

sufficiently large X and let lim([/'(x)/g'(x)] = A (finite or infinite). Define 

t = yx,x>0 and let F(f) = /(I/O , G(f) = gil/t) . Thus, F ,G are defined and 
differentiable at a right neighborhood of ? = . Moreover, 
G'(t) = g'(x)i-l/t^) ^ near / = (why?) and 

F'jt) _ f'ix)i-l/t^) _ f'(x) 
G\t) g'ix)i-\lt^) g'ix) 

which yields lim ([F'(f )/G'(?)] = A . Since L'Hospital's rules hold for the pair 
F and G , they also hold for / and g as well. 



Example 5.5.3. Let fix) = l/x , g{x) = ln(sin(x)) , a = . The functions approach 
+00 and -00 respectively as x ^ 0""" . Their derivatives satisfy 

f'ix) _ (l/x^) _ sin(x) 1 1 
g'ix) cot(x) X cos(x) X 

and the whole expression approaches -00 since sin(x)/x and l/cos(x) each 
approaches 1 as x ^ 0""" . Thus lim l/[xln(sin(x))] = -00 . 
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Example 5.5.4. Let f(x) = ln(l + x) , g(x) = x; x>0 . Here 



x^0+ 



,1-^0+ 1 



which implies that for small positive x the function ln(l + x) can be approximated 
by X (this is easily extended for negative small x) as shown in Fig. 5.5.1. 




Fig. 5.5.1. Approximating ln(l + x) (solid) by x (dashed) for small x . 



Example 5.5.5. The function f(x) = e^ approach infinity as x^oo and so is 

g{x) = X . Which one approaches faster? By repeatedly applying Theorem 5.5.3 
we get 



lim 



e 



3 



lim 



lim- 



lim- 



and this result is clearly valid for arbitrary power g{x) = x" , n>0 . 



Sometimes it is needed to estimate a function h{x) at points "near" a , when it is 
known to behave as one of the expressions Ooo,oo ,1°°,0 ,oo-oo when x 
approaches a . In such case we may try to express h(x) = f{x)/g{x) where f{x) 
and g(x) satisfy the requirements needed for applying one of L'Hospital's rules. 
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Example 5.5.6. Let h(x) = xln(x) , x > . When x ^ 0""" we obtain a behavior of 
• oo , which can be treated by choosing f(x) = ln(x) , g{x) = l/x thus obtaining 

lim xln(x) = hm ^^ = lim ^^'^\ = hm (-x) = 

.1-^0+ .1-^0+ (l/x) x^O+-(l/x^) x^O* 

Example 5.5.7. Let h(x) = l/x - l/sin(x) , x ^ . The function's behavior near 
X = is of the type oo - oo . To approximate the function for small x we rewrite 
the function as /!(x) = (sin(x)-x)/(xsin(x)) . The nominator and denominator 
satisfy the requirements of Theorem 5.5.1. Hence 

sin(x)-x cos(x)-l -sin(x) 
hm h(x) = lim = lim = hm = 

;t^o+ x^o+ xsin(x) .1^0+ sin(x) + xcos(x) x^o* 2cos(x)-xsin(x) 

Example 5.5.8. Let y = {l + 1/x)^ , x > . When x -> oo we obtain an expression 

of the type 1°° . In order to get the exact limit (which is the number e ) we firstly 
rewrite ln(}') = xln(l + l/x) and try to evaluate the limit of xln(l + l/x) as 
X ^ oo . By applying L'Hospital's rule of Theorem 5.5.2 (the reader should check 
that all the requirements hold for this application) we get 

V wi 1/ N r In(l + 1A) ,. -(l/x2)/(l + l/x) , 

hm xln(l + l/x) = hm ^ '—- = lim —-^ ^7— - — '-— = 1 

x^^ x^^ (l/x) x^~ -(l/x^) 

which implies lim y = e . 

x— >~ 

PROBLEMS 

, ^ , , , ^ ,. X -1 „ ^ ,. v.^+sin(x) ^ ^ ,. x-tan(x) 

1. Calculate: (a) lim — (b) hm -^^ —^ (c) lim ^^ . 

x^i 2x -X -1 x^Q* X a:^o+ x + sin(x) 

^ „ , , ^ ^ ,. ln[cos(x)] „ ^ ,. ln[e '^-5x1 

2. Calculate: (a) lim— !— --^ (b) lim— !^ -. 

x^O X -"-"^O X 

^ ^ , , , ^ ,. sin(x) + tan(x) „ ^ -,. sin(x) - tan(x) 

3. Calculate: (a) lim — ^-^ ^^ (b) lim — ^—^ ^—^. 

x^o ln(l + x) x^o xln(l + x) 

ln(x) x" 

4. Calculate: (a) lim —^-^ ,a>0 (b) lim — 

.1-^=0 j-a j,^„ ^x 

2 • 2 

X — sin (x) 

5. Calculate lim — r (Hint: use Taylor's theorem for sin(x)). 

x^o X sin (x) 

, _ , , ,. ln(cos(3x)) 

6. Calculate hm — ^^ ^^ — — 

x^o+ ln(cos(2x)) 
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The integral as represented in this chapter can be sometimes regarded as a 
mathematical modeling of the concept of area. However, once defined, the 
integral presents more than a tool for obtaining areas (or volumes), and calculation 
of integrals is quite popular, essential and helpful in all sciences, not just 
geometry. There are several definitions for the integral and in this chapter we 
follow most existing texts and introduce the most popular and applicable one: the 
Riemann integral. 

6.1 The Riemann Integral 

Throughout this section f{x) , unless otherwise specified, will denote a bounded 
function defined over a closed interval [a,h\ (i.e., -0° <a <b <°a). Prior to 
introducing the integral we define a partition (or mesh) of an interval. 

Definition 6.1.1. A partition (or mesh) of a closed interval {a,b'] is a finite 
collection of distinct points P = {x, }"^q such that a = Xq < jcj < ■ ■ ■ < jc,j_j < x„ =b . 
Each interval [x,_j,x,] is called subinterval and the norm of P is 
\ P\= maxjl .r, -x,_i ij. The partition's points are often referred to as mesh points 

\<i<n 

and the norm is also called mesh size. 

Example 6.1.1. Consider the interval [1,7] with the partition f = {1,2,4.5,6.5,7}. 
In this case we have 3 subintervals and the partition's norm is 2.5. 

For an arbitrary partition, the numbers 

m,. = inf [/(x)] , M,. = sup[/(x)] ; x,._i < x < x,. (6.1.1) 

exist and we may therefore define the lower and upper sums 

n n 

L{PJ) = Y^m,hx,,U{PJ) = Y,MM (6.1.2) 

1=1 /=i 
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where Ax, = x, -x,_j ,l<i<n. These sums shown in Figs. 6.1.1 and 6.1.2 are 
often called the Darboux sums and if /(x) > they provide lower and upper 
bounds to the area bounded by f(x), the x-axis and the straight lines 
x = a,x = b. Clearly, L(f,P) < U{f,P) for all f(x) and P . 

Example 6.1.2. Let /(x) = sin(x) , < x < ;r and P = {0,7i:/6,7r/3,7r/2,37!:/4,7r}. 
The lower and upper sums are 

626 26 22 2 

UiP,f)=}-.^ + ^. ^ + 1.^ + 1.^ + ^. ^^3.920 
26 26 6 2 22 

For arbitrary partitions P ,Q we may create the union R = PuQ which is a 
partition which consists of all the mesh points of P and Q . We clearly get 
P ^ R,Q ^ R , i.e., all the mesh points of either P or Q belong to R as well. 
We therefore may refer to R as a finer partition than either P or Q . The next 
result is a major step in the process of defining the integral. 

Lemma 6.1.1. Consider two partitions P = {xj}"^q and Q which consists of all the 
mesh points of P and a single additional point ^ : Xi^_^ <^ < Xj^ . Then 

UPJ)<L{QJ),U{QJ)<U{PJ) 

Proof. 

By definition 

n k—\ n 

L{P,f) = y m,Ax; = y ni^hx^ +mi^Ax^. + y nijAx^ 

i=\ /=1 i=k+l 



n k—\ n 

UQJ) = ^miAxi =^m,.Ax,. +m,n(^-x,t_i) + m,t2(x,t-^) + ^m,.Ax,. 

!=1 1=1 i=k+l 

where m^. j = inf [/(x)] , Xj,_i < x < ^ and m^, 2 = inf [/(x)] , ^ < x < x^, . Since 
m^. <mi^i,mi^2 (why?) we obtain 
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and consequently L{P,f)<L(QJ) . Similarly U(Q,f)<U{P,f) which 
completes the proof. 

Corollary 6.1.1. If P,Q satisfy P^Q, then L(P,f) < UQ,f) , U(Q,f) < U(P,f) , 
i.e., refinement of the mesh increases the lower sum and decreases the upper sum. 

The proof by repeatedly applying Lemma 6.1.1 is left as an exercise for the reader. 

Lemma 6.1.2. The relation L{PJ) < U(Q,f) holds for arbitrary P,Q . 



Proof. 

Let /? = Pug. Then L(P,f)<L(R,f)<U(R,f)<U(Q,f) which completes 
the proof. Thus an arbitrary lower sum is less than (or equal to) every upper sum. 

Since the lower and upper sums are bounded, we may define the definite lower 
and upper Darboux integrals of a function f(x) , a < x<b as 



J/(xMx = sup[L(P,/)] , jfix)dx = mf[U{P,f)] 

a a 

h h 

respectively, where the relation /(x)(ir< /(x)(ix always holds. If the two 

a a 

h 

integrals are the same, the common value denoted by {D)\ f(x)dx is called the 

a 

definite Darboux integral over [a,b'\ , the function is said to be Darboux 
integrable. 

Example 6.1.3. The function f{x) = C , a< x< b satisfies m, = M, = C for all 
possible partitions. Therefore, L(/,P) = C/(/,P) = C(/7- a) for an arbitrary 
partition P which yields 

b b 

Cdx = Cdx = C{b — a) 



a a 
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Intuitively, one would expect the lower and upper Darboux sums to converge to 
the lower and upper Darboux integrals respectively, when the norm 
I P 1= max{l X, - x,_i \] approaches 0. This is established by the following results. 

\<i<n 

Theorem 6.1.1. For arbitrary e >Q a ^ > can be found such that 

b 

\P\<d^ L(P,f) >{f(x)dx-£ 

a 

Proof. 

By definition, we can find an n- point partition 2={-^,}/Lo ^^''^ '■'^^'• 

b 

L(Q,f)> \ f{x)dx- ejl. Consider an arbitrary m- point partition ^ = {3',}™o 

a 

with norm less than some given d>Q . The union partition R = PkjQ which has 
no more than m + n-2 mesh points satisfies 

L(P,f),L(Q,f)<L(R,f) 

and consequently 

b 

LiR,f)>{fix)dx-£/2 (6.1.3) 

a 

Given two consecutive mesh points }',_i,}'iOf P there are two possibilities: (a) 
The subinterval [}',_!,)',] has no points of Q . In this case the term m,(}', -}',_i) 
appears in both L(P,f) and L(/?,/) . (b) There are k points Zi,Z2,---,Zi^ (A:>1) 
between }',_i and yj which belong to Q and to R but not to P , i.e., 
yi-i < Zi < ^2 < " " ■ < -Zyt < >'(■ ■ Denote Zq = }',_i , z<.+i = }',■ . Then, the contribution 
of[^,._i,y,.]toL(/?,/)is 

A = m^di - Zo) + '«;,2(Z2 - Zi) + •■• + wi/.i+iCz^+i - z*) 

where nij • = inf [f(x)] , z,_j < x < z, satisfies nij ^m^ j ,l< j < k + 1 . The 
difference between A, and the term wi^Cy, -}',_j) is 

Bi = Km - ot,. )(Zi - Zo ) + • • • + ('"i.A+i - mi KZk^i - Zk ) 
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and consequently I B, l<2M(}', - y,_[) < 2M(J where M denotes any bound of 
I /(x) I over [a,b] . Since the number of such subintervals in P (with interim 
points in Q) is no more than m (why?) we conclude that 

L(P,f)>L(R,f)-2mMS (6.1.4) 

The proof is completed by choosing S = ejiAmM) , and applying Eq. (6.1.3). 
The next result can be similarly proved. 

Theorem 6.1.2. For arbitrary e>0 a ^ > can be found such that 

b 

\P\<S^U{PJ)<[f{x)dx + e 

a 

Thus, the lower and upper Darboux sums indeed converge to the lower and upper 
Darboux integrals as maxjl x, - x,_i l| — ^ . 

\<i<n 

Consider an arbitrary bounded function f{x) and a partition f{x) of [a,b] . 
Let g- denote an arbitrary point within [x,_i , x, ] and generate the Riemann sum 

n 



which depends on the particular P . 

Definition 6.1.2. If lim^,, =5 as max(x, -x,_i) ^Othe function f{x) is said 

l<i<n 

to be Riemann integrable, having the definite Riemann integral S which is often 

h 

denoted by {R)\f{x)dx. In this expression /(x)is called the integrand 

a 

while a,b are the lower and upper limits of the integral respectively. The symbol 
dx is a reminder of Ax and of the fact that the integration is with respect to the 
variable X . 

Another form of Definition 6. 1.2 is: If for arbitrary f > , a d>Q can be found 
such that |y - 5,, I < £■ for all partitions with norm less than d and all choices of 
interim points ^, : x,_j < ^,- < x, ,\<i<n , f(x) is Riemann integrable and its 
Riemann integral is / . 

Since L{P,f) < S„ < U(P,f) for arbitrary /(x) , we clearly get 
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\fix)dx<S„ <[f(x)dx 

a a 

Therefore, if the Riemann integral exists, it must satisfy 

h h h 

[f{x)dx<{R)[f{x)dx<[f{x)dx 



(6.1.6) 



We thus concluded two different definitions for the integral, but apparently, if 
they exist they provide the same value, as stated and shown next. 

Theorem 6.1.3. Let f{x) denote a bounded function over a bounded closed 
interval [a,b'\ . Then, its Darboux integral exists if and only its Riemann integral 
exist as well. Furthermore, if the two integrals exist, then 



4n..-..U-.. 



(6.1.7) 



Proof. 



(a) Assume f{x) is Darboux integrable and let e >0 . Denote / = (D) f(x)dx . 

a 

Then, by virtue of Theorem 6.1.1, a ^i > exists such that 



^m,.Ar,. >/ 



£ 

2 



for any partition P ={xi}"^q with norm less than S^ . Similarly, by Theorem 6.1.2, 
a ^2 > exists such that 



1=1 ^ 

provided that I P kcJj . Consequently, 

n n 



i=\ 



i=\ 



<e 
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for arbitrary P with norm less than S = min(^j , ^2 ) ■ However, for such partition 
and for any particular choice of interim points ,^, : x,_i < 1^, < x, ,l<i<n, the 
Riemann sum of Eq. (6.1.5) satisfies 

n n 

^m,A.,.<5„<^M,.A., 

and therefore also yields |5'„-/|<£', i.e., f(x) is Riemann integrable and its 
Riemann integral equals its Darboux integral / . 

(b) Let /(x)be Riemann integrable over [a,b] , denote its Riemann integral by 
J and let £>0. By Definition 6.1.2, a ^>0 can be found, such that for 
any partition with norm less than S and arbitrary choice of 

^j : x,_i < ^,- < Xj ,l<i<n, the inequality 

J-£<S„<J + e (6.1.8) 

where S„ is the Riemann sum of Eq. (6.1.5), holds. Let us choose, for such 
partition P , a set of interim points K^ = {^, ,1 < / < n} such that 

/(^,.)<m,.+— ^,l</<« 
(b-a) 

This yields 

S„<L{P,f) + £ 

and by applying Eq. (6.1.8) we get J -e<S^ < L{P,f) + e which leads to 
J -2e < L(P,f) . Similarly, by choosing another set of interim points, we can 
show J + 2e>U(P,f). Therefore, 

U(P,f) <J + 2e< (L(P,f) + 2S + 2£) = L(PJ) + As 

which yields inf [[/(/",/)]< sup[L(P,/)] + 4£- . Since this holds for all e>Q 

P p 

we get inf[U{P,f)]<sup[L(P,f)]. However, we always have 

P p 

M[U{P,f)]>sup[L(P,f)] which yields M[U{P,f)] = sup[L(P,f)], i.e., the 
P p P p 

Darboux integral exists and by Eq. (6.1.6) it must equal the Riemann integral 

of fix) . 



154 6 Integration 

Example 6.1.3. Let f(x) = x ,0<x<2. Consider an evenly spaced partition 
with n subintervals. Tiiis lower and upper Darboux sums are 

L(P,f) = h[0^ +h^ +... + (2-hf] , U(P,f) = h[h^ +... + (2-hf +2^] 

where h = 2/n and [/(P, /)-L(P,/) = 4/z = 8/« . Therefore, f(x) is integrable 
(by both Darboux and Riemann) (why?) and its integral is (for example) 

fx^^x=limL(/',/)=limJ-| -I (l^ +2^ +... + (M-l)^)i = - 
J h^Q n^^l n\n) 3 



Deriving the last limit is a simple algebraic result and is left for the reader. 
Example 6.1.4. Define f{x) as follows: 

, X rational , < x < 1 



fix) , 

II , X irrational , < x < 1 

Since at every subinterval of [0,1] one can find infinite rational and irrational 
numbers, we clearly get 



1 » 

\f{x)dx = Q , \f(x)dx = l 



Thus, both the Darboux and the Riemann integrals of f{x) do not exist. The 
lower Darboux integral is , while the upper Darboux integral equals to 1 . 

We close this section with a result that can be applied as an integrability test 
and whose proof is left as an exercise for the reader. 

Theorem 6.1.4. A function f{x) is integrable if and only if for an arbitrary e>0 
a partition P of the interval {a,b'\ can be found such that U{P, f) - L{P, f) < e . 

PROBLEMS 

1 . (a) Show that the Riemann integral of the step function 

fl,0<x<l 



fix) , 

2,l<x<3 



exists and calculate its value. 
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(b) Redefine /(I) = 2,3,10. Does the integral still exist? Does its value 
change? 

2. Let f{x) = l/x , 1 < X < 2 and consider an evenly spaced partition P = {xi}"^Q 
of the interval [1 , 2] with m = 10 . Calculate L{P, f),U{P,f). 

3. In the previous example calculate U(P,f)-L(P,f) for arbitrary n. Does 
the Darboux integral exist? What about the Riemann integral of f{x) ? 

4. In Example 6. 1 .4 redefine 

I 1 , X rational , < x < 1 
fix) = 

[0 , X irrational , < x < 1 

Does the integral of /(x) exist? 

5. Show that 

f sin(l/x) , < X < 1 
[ , x=0 

is not integrable over [0,1]. 

6. Let a = Xy < Xj < . . . < x„ =b and consider the function 

fix) = Ci , Xj_[ < X < X; ;0 < ( < n 

b 
Show that /(x) is Riemann integrable, calculate f{x)dx and show that it 

a 

does not depend on the function's values at the nodes x, , < / < « . 
6.2 Integrable Functions 

The set of all integrable functions over a given bounded interval / = {a,b\ does 

not include all bounded functions defined over / as seen from Example 6.1.4. 
However, this set is apparently far larger than other familiar sets such as the set of 
all differentiable or continuous functions. This section provides basic results 
which establish several sujficient conditions for integrability. 

Theorem 6.2.1. A continuous function /(x) over a bounded interval I ={a,b] is 
Riemann integrable. 
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Proof. 

Since f{x) is also uniformly continuous, given £>0 one can find a S>0 such 
that 

\ti-t2\<S,a <fi,f2 ^b^>\f(ti)-f(t2)\<e/{b-a) 

Now, consider any partition P with norm less than S . The continuity of f{x) 
guarantees the existence of y^ , z, within any subinterval [x,_i , x, ] of P , such that 

m, =/(y,),M,=/(z,) 

which leads to 



U(P,f)-LiP,f) = y[f{z,)-f{y,)]Ax,<-^yAx,=£ 

and since this true for all £ >0 , f{x) is Riemann integrable which completes the 
proof. 

Example 6.2.1. The functions x , e^, sin(x) and 1/(1 + x ) are all Riemann 
integrable. So is the function 

fxsin(l/x) , < X < 3 
I 0,x = 

which is continuous over the closed interval [0,3] . 

Note that discontinuity of f(x) at a single point, a finite number of points or 
sometimes even at an infinite number of points, does not rule out the possibility 
that f{x) is integrable. For example, let Xq denote a positive integer between 
and 1 and define define 

\l,Q<X<\,Xi^Xr, 
[ , X = Xq 

This function has a single discontinuity at x = Xq , yet it is integrable. The proof is 
left for the student. 

Corollary 6.2.1. An arbitrary differentiable function over a bounded interval is 
also integrable. 
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The proof is simple: a differentiable function is also continuous and hence 
integrable. 

Theorem 6.2.2. Let f{x) denote a monotone bounded function over a bounded 
interval / = [a,b] . Then f{x) is integrable over / . 

Proof. 

Consider a monotone increasing f{x) and let £• > . If maxAjc, < S then 

n n n 

^(M,. -m,.)(x,. -x,._i) = ^[/(x,.)-/(x,_i)]Ax, < J^[/(x,.)-/(x,._i)] 

i=l !=1 /=1 

<d[f(b)-f{a)] 

and the choice J< ff/[/(fo)-/(a)] guarantees U(P,f)-L(P,f)<e, i.e., 
f{x) is integrable. The proof for a monotone decreasing function is similar and is 
left for the reader. 

Example 6.2.2. The function 

x,0<x<l 



f(x)- 



JJ+i ,l<x<3 



is monotone increasing over the interval [0,3] and therefore integrable. Thus, the 
jump at X = 1 does not affect the function's integrability. 

Example 6.2.3. The integral of 

^, , [Vxcos(l/x),0<x<l 
f(x) = \ 

[ 0,x = 

exists, since the function is continuous everywhere, including at x = . 

PROBLEMS 

1 . Why are the following functions Riemann integrable: 

a. f(x) = y-,-l<x<2 (b) f(x) = \x\ + x,-2<x<5 



0.1 + x 
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X , 0<x<l 
b. /(x) = <j 2,l<x<3 

2 + ^[x ,3<x<l 



2. Show that 



/W^ 



0,0<x<l,x^l/2,l/3 
l,x = l/2,l/3 



is Riemanti ititegrable. 
3. Define 



/W^ 



[1/2" , 1/2"-'' < X < 1/2" ; M = 0,1,2, 



0,x = 



and show the integrability of this function. 

4. Let f{x) be a bounded integrable function over a bounded interval [a,b\ . 

Show that by changing the function's definition at a single point, keeping it 
bounded, the integral still exists and maintains the same value. Can you extend 
the result for any finite set of changes. 

5. Is the function 



fix)- 



sm| — \,Q<x<7r 

X , 



0,x = 



Riemann integrable? Why? 

6.3 Basic Properties of tlie Riemann Integral 

Theorem 6.3.1. Let f{x),g{x) denote bounded Riemann integrable functions 
over a bounded interval [a,fc].Then 



u u u 

^{f{x)±g{x)]dx = ^f{x)dx±^g{x)dx 



(6.3.1) 
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Proof. 

Let £■ > . Since f{x) is Riemann integrable, there exists a Jj > such that 
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n " 

2]/(^,)Ax,-|/(x)^., 



for arbitrary partition P = V^il-c, and provided that max Ax < ^, . Similarly, a 

' l<!<n 



^2 > can be found such that 



n " 



for any partition and arbitrary interim points which satisfy max Ax, < 82 ■ Thus, if 

l<(<n 

we confine ourselves to partitions P = jx }" „ with max Ax < S = min(^, ,82) , we 

' l<i<n 

get for arbitrary choice of ^^ : X;„i < ^^ < x,- ,l<i<n, 

" i i „ i 

Y,[f(^i) + 8(^i)]^i -\f(x)dx-\gix)dx < X/(^-)^/ -\f(.^)d^ 



i=l 



+ 



n " 



<e 



Therefore, the Riemann integral of /(x) + g(x) exists and equals the sum of the 
single integrals as stated. The second part, i.e., showing the existence of the 
integral of /(x) - g{x) and calculating it, is similarly done. 

Theorem 6.3.2. Let /(x) denote bounded Riemann integrable functions over a 
bounded interval {a,b] . Then, for arbitrary constant C 



u u 

\cf{x)\dx = c\ f{x)dx 



(6.3.2) 
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Proof. 



For any partition P = jx, }"^q 
^; : x,._[ < ^,. < X; , 1 < ; < M we have 



with arbitrary interim points 



Jc/(^,.)Ar,.=cJ/(^,.)Ar,. 



(=1 



i=i 



Consider an arbitrary f > . Since f{x) is integrable, a J > can be found such 
that 



n " 



provided that max Ax, < 5 . Consequently, 

l<i<n 



n " 

^c/(^,.)Ax,. -c^f{x)dx<e 



, max Ax, < S 

l<i<n 



u 

which yields that the integral of cf{x) exists and equals to c\f{x)dx. 

a 

Theorem 6.3.3. Let f{x),g{x) denote bounded Riemann integrable functions 
over a bounded interval [a,b] such that f {x) < g{x) , a < x < b . Then 



u u 

f{x)dx< g{x)dx 



(6.3.3) 



Proof. 

For an arbitrary partition ^ = {x,}"^0 let m^,m^ denote inf[/(x)], x,_j < x < x, 

and inf[g(x)], x,_[ < x < x,- respectively. Assume m,- > m^ , i.e., m,- = nij + f,, for 

some f„ >0. Let ^'■x^_^ <^<x^ be such that g{^)<m^ +£"(). This leads to 

contradiction since f {^) < g(^) < nij + Sf^ = nii . Consequently, »?,< wi; which 
clearly yields 
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b b 

\f(x)dx<\g(x)dx 

a a 

Since both functions are Riemann integrable, we finally get 

b b 

f{x)dx< g{x)dx 

a a 

which completes the proof. 

Theorem 6.3.4. For arbitrary integrable function f{x) the integral of |/(x)| 
exists and 



|/(xMx<||/(x)|c/x 



(6.3.4) 



Proof. 

First we must show the integrability of |/(x)| . Let 



/W,/(x)>0 j 0,/(x)>0 

/+ \x) = \ , f_ (x) = J. 

^ 0,f(x)<0 \f(x),fix)<Q 



Clearly, f(x) = f^(x) + f_(x) and \f(x)\ = f^(x)-f_(x). Consider a partition 
P = {xj }"^y and define 

mf = inf [/+(x)] , Mr = sup[/+(x)] ; x,._i < x < x,. 

The integrability of /+(x) is established as follows. If /+(^) > for some ^^e /„ 
then M^=Mi. Obviously m'l' >mi which yields M/" -m/" < M,- -m,- . 
However, this inequality holds even if such ^e 7,^ does not exist, since then 
f+{x) = , x,_[ < X < X,. and consequently m/" =M^ =0 which yields 
= M^ - m/" < M; - m, . Generating the lower and upper sums we obtain 

U(P,f^)-L(P,f^)<U(P,f)-L(P,f) 
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and since f(x) is integrable, so is /+(x) . The relation f_(x) = f{x) - /+(x) and 
Theorem 6.3.1 yield that /_(x) is integrable as well and the relation 
|/(x)| = f^{x) — f_{x) combined with Theorem 6.3.1 establishes the integrability 
of |/(x)| . Since ± f{x) < |/(x)| , we may apply Theorem 6.3.3 and get the 
inequalities 



±^f{x)dx<^\f{x)\dx 



which complete the proof. 

Theorem 6.3.5. Let f{x),g{x) denote bounded Riemann integrable functions 
over a bounded interval [a,b\ . Then, the Riemann integral of h{x) = f(x)g(x) 
exists. 

Proof. 

Unlike in the Theorems 6.3.1 and 6.3.2, we do not have a simple specific value for 
the integral of h(x) , which is solely based on the integrals of f(x) and g(x) . 
Therefore, the only reasonable way to show its existence is to compare between 
the lower and upper sums of h(x) for an arbitrary partition P = jx, }"^q . Let 

m-i = inf [/(x)] , M,. = sup[/(x)] ; x,_i < x < x,- 

m* = inf [g(x)] , M* = sup[g(x)] ; x,._i < x < x,- (6.3.5) 

mj" = inf[h(x)] , MJ = sup[h(x)] ; x,_j < x < x,- 

We distinguish between two cases: 

(a) /(x)>0,g(x)>0. Denote M =sup[/(x)] , M* =sup[g(x)] ;a < x<Z7 . 
Then 



L{P,h) = /.^i ^i - y.^i'^' ^i 

i=l (=1 

n n 

U{P,h) = ^M*'Ax,. <^M,.M*Ax,. 
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leading to 



U(P,h)-L{P,h) < ^(M,.M* -m,.m*)Ajt:,. 



1=1 1=1 

<M ■[U(P,g)-L(P,g)] + M' -[UiPJJ-LiPJ)] 

Given an £■ > we can find a ^j > and S2 >0 such that 

U(P,g)-L(P,g)<e/(2M),\p\<Si 

U(P,f)-L(P,f) < £/(2M*) , \P\ < S2 

and thus U(P,h)- L(P,h) < e provided that |p| < ^ = min(^, , ^2 ) • This 
completes the proof of (a). 

(b) General f(x),g{x). Find constants A,B such that F{x) = f(x) + A>0 
and G{x) = g{x) + B>0 . Since any constant is a Riemann integrable 
function, so are the functions F(x) and G(x) (why?). Also 

f(x)g{x) = [F{x) - A][G{x) -B] = F{x)G{x) - AG(x) - BF{x) + AB 

Thus, the left-hand side is obtained by adding and subtracting four Riemann 
integrable functions and must therefore be itself Riemann integrable. 

Theorem 6.3.6. Let f{x),g{x) denote bounded Riemann integrable functions 
over a bounded interval {a,b] and let inf[g(x)]>0. Then, the function 

a<x<h 

h{x) = f(x)lg{x) is also Riemann integrable over [a,b}. 

Proof. 

The proof is left as an exercise for the reader. 

Theorem 6.3.7. If f{x) is integrable over [a,b\, then it is integrable over any 
subinterval of {a,b\. 
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Proof. 

Let a <c < d <b . For clarity let the lower and upper sums defined by Eq. (6.1.2) 
be re-denoted by L(P,f,a,/3),U(P,f,a,^) respectively, where [«,/?] is any 
subinterval of [a,b]. We need to show that given an arbitrary f > , a S>0 can 

be found such that for any partition P = jx, }"^q with IpI < J , 

U(P,f;c,d) - L{P,f-c,d) < e 

Since f{x) is integrable over [a,b\, we can find a (^j > such that for any 
partition 2={}'i}"Io ^^ ['^'^1 with IqI < Jj , the inequality 

U{Q,f;a,b) - L{Q,f\a,b) < e 

holds. Let P = {xi}"^Q denote a partition of [c,d] with 1^1 < ^i and extend it to 
any partition P'over [a,b], provided that |P'| < Jj . The special choice of ^j 
guarantees 

U(P',f;a,b)-L{P',f;a,b) < e 

Also, 

U(P,f;c,d)-L{P,f;c,d)<U{P',f;a,b)-L(P',f;a,b) 

since P is included in P' . Therefore U(P,f;c,d)-L(P,f;c,d)<£ for any 
partition with norm less than Jj and the proof is completed. 

Theorem 6.3.8. If f{x) is integrable over [a,b] and over [b,c], then it is 
integrable over [a,c] and 

c b c 

\f(x)dx=\f{x)dx+\f{x)dx (6.3.6) 

a a b 

Proof. 

By Theorem 6.1.4 there is a partition P of [a,b] such that 

U{PJ;a,b)-L{PJ;a,b) < - 
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and another partition Q of [b, c] such that 

UiQ,f;b,c)-L(Q,f;b,c)<^ 

The partition R = PvjQ is clearly a partition of [a,c] such that 

L{R,f;a,c) = L{P,f;a,b) + L(Q,f;b,c) 
U(R,f;a,c) = U{P,f;a,b) + U(Q,f;b,c) 

which consequently yields 

U{R,f;a,c) - L{R,f;a,c) < e 

and by Theorem 6. 1 .4 the proof is concluded. 

We have previously established that all continuous functions C[a,b] and all 

monotone functions M[a,b] are also integrable (Theorems 6.2.1 and 6.2.2), i.e., 

belong to R[a,b] - the set of all Riemann integrable functions over [a,b] . 

Another important subset of R[a,b] is the class y[a,Z7] of all functions with 

bounded variation (see 4.5). 

Theorem 6.3.9. Let /(x) denote a function with bounded variation over the 
interval [a,b] . Then f(x) is Riemann integrable. 

Proof. 

By virtue of Theorem 4.4.4 f(x) can be represented as a difference between two 
monotone functions. We now apply Theorems 6.2.2 and 6.3.1 and conclude that 
f{x) is Riemann integrable. 

We close this section by extending the definition of the Riemann integral to 
include the case b <a and to simplify the proof of the fundamental theorem of 
calculus (see below). 

Definition 6.3.1. Let f(x) denote a Riemann integrable function over [a,b] 
where a <b . We define 

a ha 

\f(x)dx = -\fix)dx , \fix)dx = (6.3.7) 
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Corollary 6.3.1. Theorem 6.3.8 holds for arbitrary a,b,c . The proof, based on Eq. 
(6.3.6), is left for the reader. 

PROBLEMS 

1. Let f{x) denote an integrable function over the interval [a,b] and let 

a = Xq < X[ < ■ • • < x,j = b 
Use induction to show 



\f(x)dx = '^ \f(x)dx 



2. Consider a continuous function f(x) and an increasing monotone function 
g(x) over [a,b] and let g(a) = 0.001 . Are the functions 

f(x) + g(x), f(x)g(x) and f(x)/g(x) integrable? 

3. Let /|(x),/2(x),...,/„(x)be integrable over the interval [a,b]. Show that the 



function F(x) = I l/,(x) is also integrable. 



i=i 



4. Show that the integral of f(x) = r- , - 1 < x < 4 exists. 

I — x + X 

^ „, , ,'cos(x)dx 

5. Show that --^ — exists. 



[■ cos(x)^ 
J 10"*' + 



X 



6.4 The Fundamental Theorem of Calculus 

Let f{x) be integrable over the interval [a,b] . By virtue of Theorem 6.3.5 it is 
integrable over any subinterval [a,x] , a< x<b . We define 



F(x)=^f(t)dt (6.4.1) 



as the Riemann integral of the function f(t) over the interval [a,x] . This integral 
which is a function of x is called an indefinite integral of f(t) . Since we chose 
the letter x as the independent variable of the indefinite integral F(x) , we 
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replaced the independent variable of the integrand in Eq. (6.4.1) by / . The basic 
features of the indefinite integral are given next. 

Theorem 6.4.1. The function F{x) defined by Eq. (6.4.1) is continuous over 
[a,b] . 

Proof. 

For any fixed XQe[a,b] and an arbitrary x, we may apply Corollary 6.3.1 
and get 

X 

Fix)-FixQ) = ^fit)dt (6.4.2) 

Since f(t) is bounded over [a,b] , there exists M >0 such that \f(t)\ < M for 
all / and consequently, by using the definition of the Riemann integral, we obtain 

|/='(x)-/(xo)|<M|x-Xo| 

and the continuity of F(x) is straightforward. 

The differentiability of F(x) is not guaranteed except in the case discussed 
next. 

Theorem 6.4.2 (The fundamental theorem of calculus). If f(t) is continuous at 
Xq then F(x) has a derivative at Xq and 

F\xo) = f(xo) (6.4.3) 

Proof. 

Using the standard terminology, we need to show 

F(xQ+h)-F(Xfi) 

hm = '-^ ; ^ = /(Xo) 

h^O h 

i.e., for arbitrary f > there is a S>0 such that 

F{xQ+h)-F{xQ) 



/Uo) 
h 



<e 
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b 
If 

provided that /; < ^ . By Example 6.1.3 C = Cdt for an arbitrary constant 

h — aJ 



C and hence, by virtue of Theorem 6.3.5, 



F{xQ + h)-F(xo) 



■/(-^o) 



x„+h 



^ \f(t)dt-f(xo) = ^ \[f(t)-f(xo)]dt 



Xf,+h 



x„+h 



dt 



The continuity of / at Xg guarantees the existence of ^ > such that 



\t-Xo\<S^\f{t)-f(xo)\<- 



Thus, by applying Theorem 6.3.4, if \h\ < d then 



F(xQ+h)-F{X(f) 



h 



fi^o) 



h 2 



which completes the proof. 

Theorem 6.4.2 provides us with a tool for calculating integrals. Recall the first 
mean-value theorem (Theorem 5.2.7) which stated that an arbitrary differentiable 
function f(x),a<x<b satisfies f(b)-f{a) = {b-a)f'{^) for some interim 
point ^ :a <^ <b . An immediate consequence is that if f'{x) = , a< x <b , 
then f{x) is constant over the whole interval (the proof is left as an exercise for 
the student). Now, let 



F(x)=jf(t)dt 



where f(t) is continuous everywhere, and assume the existence of some 

G{x) , a < x<b , called a primitive of f{x) , which satisfies 

G'(x) = f(x) , a< X <b . By Theorem 6.4.2 F'{x) = f{x) and therefore 
{F(x) - G{x))' = f(x) - f{x) = . Consequently 



F{x)-G{x) = C ,a<x<b 
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where C is a constant. By substituting x = a we get C = -G(a) and since 
F(a) = obtain F(x) = G(x) - G{a) . Thus, in order to get the integral F{x) , it is 
sufficient to find any G{x) with the property G'(x) = f{x) and calculate 
G{x) - G{a) . 

Example 6.4.1. Let 

X 

F(x)=\t''dt 
1 

Since the function G{x) = x"""" /(« + 1) satisfies G'{x) = x" everywhere, we obtain 
F{x) = G{x)-G{a)-. 



x"^'-l 



n + \ 
Example 6.4.2. Let 



F{x) = cos{t)dt 



The integrand is a derivative of G{x) = sin(x) and therefore F(x) = sin(x) since 
here G(0) = . 

If for some f(x) , a method to obtain G{x) cannot be found, we have no choice 
except approximating F(x) by taking a fine partition and calculating a lower and 

upper sums, such that the difference between them is below a prefixed degree of 
accuracy. 

We usually denote an arbitrary primitive by F{x) and express the definite 
integral as 

b 

^fix)dx = [Fix)l = F(x)l = Fib) - F{a) 

a 

Since F(x) + C is also primitive for an arbitrary constant C , we often write 
f(x)dx = F(x) + C as a "definition" of the indefinite integral, when in fact it is 
simply the set of all primitive functions of f{x) . 
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The length of a curve 

We previously introduced the definite integral as mathematical modeling of the 
concept of area. We close this section by showing that it can also be used for 
modeling another concept: the length of a curve. For the sake of simplicity we 
restrict ourselves to the curve created by a single differentiable function 
y = f(x) ,a< x<b such that f\x) is continuous. 

Let P : a = Xq < x^ < . . . < x^ = b denote an arbitrary partition of the 
interval \a,b\. Let Ax^j, = x^. - Xj._i , Ay^. = yf, - }'^_i ;\< k < n where 
yf. = f{xi^) ,0< k <n . Clearly, by using the Pythagorean theorem, the sum 



5p = £ ^Ax', +Ayl=^ 1 + -^Ax, (6.4.4) 



k=l k=\ V ^k 



can be easily accepted as an approximation to the "length" of the curve defined by 
the set of points {(x,/(x) , a < x < Z?)}. The continuity of f'{x) guarantees 



lim Sp = {Jl + [f\x)fdx (6.4.5) 

it,,,l<i<n J 



and the integral at the right-hand side of Eq. (6.4.5) is defined the length of the 
given curve. 

1 
Example 6.4.3. The length of y = 2x,0<x<l is Nl + 2^dx = V? while the 



2 

length of y = x^ ,0<x<l is U[ + 36x^^dx . 

1 

PROBLEMS 

1. Find the following definite integrals: 

332 2 

(a) — dx (b) [x-sin(x)](ix (c) dx 

1 1 

2. Find the following indefinite integrals: 
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(a) \ — dt (b) \— dt (c) \[sm(t + l) + cos(t) + —]dt 

J l + j2 J f J f2 

3. Let 

jc,0<jc<l 



2-x,l<x<2 



3/2 

Find a primitive function of f{x) if there are any and calculate f(x)dx . 


It 

4. Find the definite integral [sin(5x)cos(3x)](ix:. 



5. Calculate (a) (b) (x + 2) Jx (c) jc cos(x )dx 



6. Find the primitive functions of (a) xe (b) sin(x)cos(x) 



6.5 The Mean- Value Theorems 

Like in differential calculus there are several mean-value theorems for integrals 
which provide some information related to an integral's value without actually 
calculating it. The first result is a general mean-value theorem which is practically 
equivalent to the first mean-value theorem for derivatives (see Section 5.2). 

Theorem 6.5.1. Let /(x) denote a Riemann integrable function over the interval 
\a,b\ with m= inf [/(jc)],M= sup [/(x)] . Then, there exists A : m < /I < M 

a<x<.h a<x<Jb 

such that 

h 



\f{x)dx = X{b-d) (6.5.1) 



Furthermore, if f(x) is continuous then X = f(^) for some ^ : a< ^ <b . 

Proof. 

The double inequality 
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b 

m(b -a)< \f(x)dx <M(b-a) 

a 

clearly holds (why?) and therefore 

b 

^f{x)dx 

m<- <M 

b-a 

h 

which provides the choice A,= \ f(x)dx/(b-a) . If f(x) is continuous every 

a 

value between m and M is obtained at least once by / , which completes the 
proof. 

Theorem 6.5.2 (The first mean-value theorem for integrals). Consider two 
Riemann integrable functions f(x),g{x);a<x<b such that g(x) > 
everywhere and denote 

m= inf [f(x)],M = sup [fix)] 

a<x<h ^<j.<^ 

Then, there exists X:m< X< M such that 

h b 

\ f{x)g{x)dx = /i\ g(x)dx (6.5.2) 

a a 

Proof. 

Quite clearly mg{x) < f{x)g{x) < Mg{x) , a < x <b and by Theorem 6.3.3 

h b b 

m gix)dx < f(x)g(x)dx < M g{x)dx 

a a a 

b 

By dividing these inequalities by g{x)dx we obtain 

a 
b 

\f{x)g{x)dx 
m<-2—^ <M 

h 

^g{x)dx 
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and the proof is completed by choosing 



I fix)g(x)dx / J g(x)d. 



The particular case g{x) = 1 , a < x<b provides the result of Theorem 6.5.1. 

Corollary 6.5.1. If f(x) is continuous, then A of Eq. (6.5.1) can be replaced by 
f{^) where ^ is some interim point within the interval. 

Proof. The continuity of f(x) guarantees the existence of Xi,X2'-a < Xi,X2 <b 
such that m = f(xi) , M = /(X2) . Furthermore, for every a .m<a < M there is 
a /? : Xj < /? < X2 such that f{P) = a. In particular f{^) = X for some 
^ : Xj < ^ < X2 . This completes the proof. 



Example 6.5.1. Let f{x) = x^ ,\<x<3. Here 

3 

I 



\ X dx = — 



3^ 1^ 



_26 

3 3 ~ 3 



Since m = l,M=9 there exists X:\<X<9 such that (26/3) = A(3-l) . This 
yields /I = 13/3 which is obviously between 1 and 9. The continuity of f{x) 

guarantees the existence of ^ ■.\<^ <3 such that ^ =13/3. This implies 
^ = 2.08 which is indeed within the interval [1,3]. 

2 
Example 6.5.2. Consider the integral (x-1) x dx where both f{x) = {x-\) 



and g{x) = x are integrable and g{x)>Q. The integral, using previous basic 

results, is easily calculated and equals to 28/15 (this is left for the student to 
verify). Also 

2 

\x^dx = A (how?) 



and m= inf [(x-1) ] = 0,M= sup[(x-l) ] = 1. Therefore, by Theorem 6.5.2 

0<x<2 o<x<2 

there exists a /1 : < A < 1 such that 28/15 = 4/1. The last equation determines 
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X = 7/15 which is indeed between and 1. Also, since f{x) is continuous, there 
exists a ^:0<^<2 such that /(^)=7/15. This yields ^ = 0.683 which is 
between and 2. 

Theorem 6.5.3 (The first mean-value theorem for integrals). Let f(x),g(x) 

denote two integrable functions over the interval [a,b] such that 
g{x)>0,a<x<b and f{x) is monotone. Then, there exists a ^ :a <^ <b 
which satisfies 

h i b 

^f{x)g{x)dx = f{a)^g{x)dx + f{b)^g{x)dx (6.5.3) 

a a ^ 

Proof. 

Define 

t h 

' ^ 'X 



h(t) = f(a)^g(x)dx + f(b)^g(x)d. 



By Theorem 6.4.1 both integrals at the right-hand side are continuous functions of 
t , i.e., h(t) , a<t <b is continuous as well. Since g(x)>0 everywhere, we may 

h h 

apply Theorem 6.5.2 and get \ f{x)g{x)dx = X\ g{x)dx for some X between 

a a 

m= inf [/(jc)] and M= sup [/(x)] . However, f{x) is monotone which 



a<x<h 



a<x<h 



implies that X lies between f(a) and f(b) . By substituting t = a,b we obtain 

h h 

h{a) = f{b)^g{x)dx , Kb) = f{a)^g{x)dx 



u 

and clearly the number X\ g(x)dx must lie between h(a) and h(b) . Now, the 

a 

continuity of hit) implies (see Theorem 4.3.5) that for an arbitrary value r] 
between h(a) and h{b) there is a number t, : a < t, <b such that h{£,) = J] . This 

h 

is particularly true for Tj = X\ g(x)dx which yields 
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L) U U U 

h(t) = /(a) I g(x)dx + f(b)\ g(x)dx = A\g(x)dx =\f(x)g(x)dx 



and concludes the proof. 
PROBLEMS 

d 

1. Consider a continuous function f(x),a<x<b which satisfies \f{x)dx = 

c 

for all a<c<d<b. Show /(x) = 0,a<x<b. 

2. Let fix) = e\0<x<2. Apply Eq. (6.5.1), calculate A and find 

Xq , < Xq < 2 such that e^" = X . Is Xq unique? 

3. Repeat and solve problem 2 for /(x) = sin(x) , < x < ;r . 

4. Apply Theorem 6.5.2 to /(x) = sin(x ) , g(x) = x ,'d<x<n and obtain X . 

5. Apply Theorem 6.5.3 to /(x) = sin(x) , g{x) = cos(x) , < x < ;r and find ^ . 

6. Repeat and solve problem 5 for /(x) = sin(x) , g(x) = cos (x) , < x < .;r . 

6.6 Methods of Integration 

Definition 6.6.1. An arbitrary function that can be obtained from polynomial, 
trigonometric, inverse trigonometric, exponential, and logarithm functions by a 
finite number of additions, subtractions, multiplications, divisions and 
compositions is called an elementary function. 

, —X 

Example 6.6.1. The functions x sin(x) , , sin(tan(e^"'" )) are all 

tan(x) 

elementary functions. The third function is obtained by composition of 
trigonometric, exponential and polynomial functions. 

X 

It is clearly desirable to express an indefinite integral f{t)dt as an elementary 

a 

h 

function F(x), and obtain \ f(t)dt = F(b)- F(a) . Indeed, one then performs 

a 

two simple calculations to get the exact result. Unfortunately, this is usually 
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X 

impossible. For example, it can be shown that the popular integral e ' dt , which 

a 

occurs in many physical problems cannot be presented as an elementary function. 
Consequently, in most cases, one must turn to numerical methods (see 10.6) which 

h 

only approximate f(t)dt . Two methods that can sometimes be applied to obtain 

a 

an indefinite integral as an elementary function are given below. 
Integration by parts 

Let f{x) , g{x) , h{x) denote differentiable functions such that 

h(x) = f(x)g'(x) (6.6.1) 

Since (/(x)g(x))' = /'(x)g(x) + /(x)g'(x) we obtain 



A A A 

jh(t)dt = ^(f(t)g(t)ydt-^f\t)g(t)dt 



which leads to 



A A 

^hit)dt = fix)gix)- fia)g(a)-^ f'(t)git)dt (6.6.2) 

a a 

Thus, if the integral of f'(x)g{x) is an elementary function, so is the integral of 
h{x) . The relation given by Eq. (6.6.2) is called integration by parts and is 
usually written as 

^f(x)g\x)dx = f(x)g(x)-^f\x)g(x)dx + C (6.6.3) 

where C is an arbitrary constant. 

Example 6.6.2. Consider the integral xe'^dx . By writing f(x) = x , g'{x) = e^ 
and applying Eq. (6.6.3) one gets 

xe^dx = xe'^ - e^dx + C = xe^ —e^+C 

which is an elementary function. 
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Example 6.6.3. In order to express xln(x)dx as an elementary function, denote 
f{x) = \n(x) , g'{x) = X . By Eq. (6.6.3) we obtain 

2 r ^ 1 ^ ^ 

x\n(x)dx = — ln(x) dx + C = — ln(x) h C 

J 2 J 2 X 2 4 



Integration by substitution 

r dx 

Consider the indefinite integral f{x)dx and let x = g{u), i.e., — = g'{u) 

J du 



dx , 
du 
formally dx = g'(u)du . Then under certain quite general conditions 



^fix)dx = ^f(g(u))g'(u)du (6.6.4) 

b d 

^f{x)dx = ^f{g{u))g'{u)du (6.6.5) 



here a = g(c) , b = g(d) . 

While the left-hand side of Eq. (6.6.4) may be difficult to calculate as an 
elementary function, the left-hand side may be easily manipulated. This approach 
is called integration by substitution. Usually, the substitution is in the form of 
h{x) = g(u) , i.e., h'{x)dx = g'(u)du - depending on the expression defining the 

integrand f{x) . 

Example 6.6.4. Consider the integral xsin(x + l)dx . By substituting u = x +1 

we get du = 2xdx . Consequently 

•sin(M) cos(m) cos(x +1) 
du = ' 

2 2 



f . . 2 , ix , f • / ^J fsm(M) cos(M) cos(x +I)^„ 

xsm(x + l)dx = xsin{u)dx = du = \-C = h C 

J J J 2 2 2 

Example 6.6.5. In the indefinite integral ^r substitute u = x + 3 . Thus, 

J x^+3 



In the mdetmite mtegral I 

du = 2xdx , i.e.. 



2xdx C du , , s „ 1/2 



f 2xdx f di 
J x^+3 J M 



ln(M) + C = ln(x^ + 3) + C (6.6.6) 
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r f'{^) 
This example is a particular case of a more general integral dx where 

•" fix) 
f{x) is an arbitrary function with continuous derivative. By substituting 
M = fix) we get du = f'{x)dx which implies 

[^-^dx =[— = ln(M) + C = ln( /-(x)) + C (6.6.7) 

J fix) i u 



PROBLEMS 

1. Use integration by parts to calculate (a) xsin(x)(ix (b) \xe^dx 

r r xdx 

3. Use integration by parts to calculate (a) e* cos(x)(ix (b) \^—- 

1 3 

4. Use substitution to calculate (a) xVl-x dx (b) dx 



5. Use substitution to calculate 



ln{x)dx 



6. Calculate x"e^dx and use the result to obtain x e^dx . 

7. Calculate the integrals (a) \.dtnix)dx (b) 

J J xln(x) 

r dx, i X \ ( X 

8. Calculate the integral (Hint: replace sin(x) with 2sin — cos — 

J sin(x) y2j 1^2 

and use the substitution method). 

dx 



9. Use problem 8 to obtain 

f dx 

10. Calculate the integral for two cases: (ci) a^b ih) a =b . 

J (x — a)(x — b) 



cos(x) 
dx 



(x — a)(x — b) 

dx 
(x — a)(x — b){x — c) 

,,T , r.n^s, „, , cos(x)sin"" (x) « — 1 , 

12. Let /„= sm"(x)Jx. Show /„ = ^—^ ^^ + /„_2 and use the 

J n n 



C dx 
11. Calculate the integral where a,b,c are non-equal. 

J (x — a)(x — b)(x — c) 



formula to obtain /2 , ^3 • 



13. Similarly to problem 12, obtain an expression for /„ = cos"(x)dx 
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6.7 Improper Integrals 

b 

So far we dealt with the class of "proper" integrals f(x)dx , where a,b are finite 

a 

numbers and f{x) defined and bounded for all x: a< x <b . We will now 
extend this class to include other integrals called "improper integrals". 

h 

Definition 6.7.1. The expression f(x)dx is called an improper integral of first 

a 

kind if at least one of the endpoints of the integration interval is infinite, i.e., if 
a = -oo or i> = oo or both. If f{x) is unbounded at one or more points, it is called 

an improper integral of second kind. If at least one endpoint is infinite and the 
integrand is not bounded simultaneously, the integral is called an improper 
integral of third kind. 

Example 6.7.1. The expression e^'^dx is an improper integral of the first kind, 



f dx 
while —j= is an improper integral of second kind. 



r s dx. 
Example 6.7.2. The expression — -j=— is an improper integral of third kind 

since the right endpoint is infinite while the integrand is not bounded at the left 
endpoint but converges there to oo . 

Defining the value of an improper integral is straightforward. We start with the 
first kind. 

Definition 6.7.2. Consider a function f{x),a<x<o° such that a is finite and 

h 

the integral f(x)dx exists for all b : a<b <oo , If the integral 

a 

b 

I(b) = jf(x)dx 

a 

converges to a finite value / as Z? — > ©o , we write 
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b 
\f(x)dx = f = \im\f{x)dx (6.7.1) 

a a 

We similarly define the case where b is finite and a^>°° . 



oo b 

Example 6.7.3. Let / = — . For all finite b we have — =ln(x)| = XnQ?) 
J X J X ^ 



and 



X •> X 

1 1 



clearly ln(b) — > oo as b ^oo , Hence, the given integral - an improper integral of 

' dx 

X 



f dx 
first kind, does not converge. On the other hand the integral —r- exists since 



b 

C dx 1 

•> X X 



h 



I >\ asb^^ (6.7.2) 



Calculating an improper integral of second kind is given next. 

Definition 6.7.3. Consider a function f(x),a<x<b such that a , b are finite, 

h' 

f(x) is bounded everywhere except at b and the integral f{x)dx exists for all 

a 
h' 

b' : a<b' <b .If the integral f(x)dx converges as b' —>b we write 

a 

h b' 

\f{x)dx= lim \f{x)dx (6.7.3) 



The case where the integrand is not bounded at the lower endpoint is similarly 
treated. 

r dx 
Example 6.7.4. The improper integral -j= is well defined (i.e., is finite) since 



f-^ = 2Vx| =2-2V^^2 ase^O 



(6.7.4) 
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We finally define the value of an improper integral of third kind, whenever it 
exists. There are four possibilities - one is discussed in detail. The other can be 
similarly obtained. 

Definition 6.7.4. Consider a function f{x) ,-o° <a< x<oa such that f(x) is 

h 

not bounded at x = a . If the integral f(x)dx exists for all a > a , b <oa and if 

a 
h 

31* =\\m\ f{x)dx ,a' ^> a ,b ^>'^ (6.7.5) 

d 

we define / = f(x)dx . 

a 

It should be noted that in Eq. (6.7.5) a ^> a ,b ^> oo independently. 
Example 6.7.5. Let 

I = ]fix)dx = \'—^ 
^-^ 

A specific primitive function of /(x) is F{x) = -2e "^ and clearly 

lim F(x) = 
limF{x) = -2 

which implies that / is an improper integral of third kind which exists and 
equals 2. 



Example 6.7.6. Let 

/=|/(x)rfx=j^-^ 



This is an improper integral that does not converge at x = . Indeed, for 

sufficiently small x (say x<£) the integrand is larger than — (since 

2x 

lime" = 1 ). However, since — diverges as ;; ^ (why?), so is / . 
x->o J 2x 
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PROBLEMS 

1 . Show that the following integrals are convergent improper integrals of the first 
' dx 



kind: (a) [^ (b) \e'"'dx ,t>0 
1 ^ 

2. Which of the following improper integrals of the first kind are divergent: 



-0.01 

(a) j ^ (b) j— (c) ^smHx)d: 



X 
1 



3. Find which of the following improper integrals of the second type are 

1 n:/2 2 

, . • . ,• / N Cln{x)dx ,, , f dx ^ r dx 

convergent and which are divergent: (a) (b) (c) 

J X J sin(x) J ln(x) 

1 

4. Determine the kind of following improper integrals and calculate their 

2 -1 1 -^ ^ 

x^|l + x* 
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This chapter presents the concept of infinite series and provides several useful 
tests for determining whether a given series is convergent. 

7.1 Convergence 

n 

Consider a sequence {a^ }"^[ . For each n , the sum 5'„= /^fl^ is called the n- th 
partial sum of the sequence and the generated sequence {S„ }^^i defines an infinite 



series which is usually denoted by 2, ^n ■ 



Definition 7.1.1. If lim S„=S , where -oo < 5 < oo , the infinite series is 
convergent, it converges to S and the state of convergence is denoted by 

5=1; fl„ (7.1.1) 

Otherwise, the infinite series diverges. 

Example 7.1.1. The infinite series generated from the sequence {1,1/2,1/2 ,...,} 
converges to 2 , while the infinite series of {1,-1,1,-1,...,} diverges. The proof 
is left as an exercise for the student. 

Theorem 7.1.1. If ^ a„ is a convergent infinite series, then lima„ = . 



«^0 
1=1 



Proof. 



Since a„ = S„ - S„_^ we get lim a„ = lim S„ - lim S„_i =5-5 = 0. 
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Thus, the relation Hm a„ = is a necessary condition for the convergence of an 

infinite series. It is certainly not a sufficient condition as demonstrated by the next 
example. 

Example 7.1.2. Consider the harmonic series ^ — . Obviously, lim— = yet 



. , n n->o n 



since 



111 11111 

- + — >— , - + — + — + ->— etc. 
342 56782 

we easily conclude (see Example 3.7.5) that the series does not converge but 
rather satisfies lim S^ =•>= , and therefore, by Definition 7.1.1, diverges. 

A necessary and sufficient condition for the convergence of an infinite series 
was already presented by Theorem 3.7.9 (Section 3.7.3): Given an arbitrary 

sequence {fl„ jJJ'^j , the associated infinite series is convergent, if and only if 
{Sn}7=i is a Cauchy sequence, i.e., if for any given f > , an integer MqC^) ^^^ 
be found such that |5„-5'^|<f for all n,m>nQ{e). Thus, an infinite series 
converges if and only if, given f > , an integer iiq {e) can be found such that for 
any n,m which satisfy nQ<n<m, the inequality |a„ + fl„^.] +... + fl,„| < f 
holds. 

Example 7.1.3. Consider the infinite series 1 + 1/3 + 1/3^ + . . . + 1/3" + . . . . Here 

\Sm-Sn\ = |l/3"^' + 1/3"^^ + . . . + 1/3'" I = (1/3"+' )(l + 1/3 + . . . + 1/3™-"-' ) 



which leads to 



\s,-s.\4h"'p^^<- ' 



1-1/3 2-3" 



Thus, given f > we may choose Wn such that < e , which implies 

2-3"» 

, ln[l/(2g)] 

"0 > 

In 3 
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Example 7.1.4. The infinite series a + aq + ... + aq" + ... converges for arbitrary 
q .-l< q <1 and satisfies 

^a^«-i=_^ , -1<^<1 (7.1.2) 

n-\ " 

The proof, similar to that given in Example 7.1.3, is left as an exercise for the 
student. 

Although we presented an infinite series as ^a„ , quite often the sequence 

generating the series may start with a,,, where m is an arbitrary fixed integer not 

equal necessarily to 1 . Also, we may for some reason drop or ignore the first k 
terms of a series. This has usually no ejfect on the convergence or divergence of 
the series. In order to include these cases, we will usually denote a sequence by 

{«„} and an infinite series by /.'^n ■ 

Theorem 7.1.2. Consider an infinite series /.^n ^^^^ that a,^ >0 for all n. 
Then , the series converges if and only if its partial sums are bounded above. 

Proof. 

If the sequence {S„] is bounded above, it must converge by virtue of Theorem 
3.7.1, as an increasing bounded monotone sequence, i.e., lim S„=S . Conversely, 

if the partial sums converge, the sequence {S^} is obviously bounded. 

The reader may notice that Theorem 7.1.2 holds even if a„ > only for all 
« > Mq for some fixed n^ (why?). 

Theorem 7.1.3. If / a„ converges so is y.^'^n for arbitrary constant M . 

The proof can be found in Section 3.5 (In the proof of Theorem 3.5.1 choose 
b„=M for all n ). 

Theorem 7.1.4. If /^a„ and /^^„ converge, then 2_,(^n-K) converge as 
well. 

The proof, using Cauchy's criteria is left as an exercise for the reader. 
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PROBLEMS 

1 . Is the series generated by {l , , 1/2 , , , 1/4 , , , , 1/8 , . . .} convergent? 

2. Is the series generated by {1,0,1/2,0,0,1/3,0,0,0,1/4,...} convergent? 

3. An infinite series /_^a„ is such that a„ > ,n>nQ , its first 20 partial sums 
are bounded by 1000 and the rest of them are bounded by 5 . 

(a) Does the series converge? 

(b) What can be said about S = ^ fl„ ? 

4. Find whether / ^a„ converges in the following cases: 

1 1 2* 1 

(a) a„ = (b) a = (c) a = (d) a„ = —j= 

3+2 3-2 3-2+K -sin 

7.2 Tests for Convergence 

In this section we present several useful tests for convergence of infinite series. 
We start with the comparison test which applies the already known convergence 
of one series for determining the convergence of another. 

Theorem 7.2.1 (Comparison test). Let /^fl„ and /^fc„ denote two infinite 
series such that 0<a„ <i'„ for all n. Then, the convergence of /.^n implies 
the convergence of ^a„ and the divergence of ^fl„ implies the divergence 

of Z^« • 

Proof. 

Denote the partial sums of two series by [S ^ „ } and {Si,„} respectively. Then 
clearly Sg,^<S^,,^ for all « . If lim 5^1, „ = 5/^ then {S,,^} is bounded above and 

n— >«> 

SO is {S^ „ } . Therefore, by Theorem 7.1.2, the sequence {S^ „ } is convergent. On 
the other hand, if {5„„} diverges and [S),,^} converges, we obtain contradiction 
to this theorem's first part. Thus, {5^ „ } must also diverge. 

As in Theorem 7.1.2, the result holds even if < a„ < fo„ only for almost all n , 
i.e., for all n>nQ where Mq is any fixed positive integer. The proof is left for 
the reader. 
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Example 7.2.1. The infinite series > — converges. Indeed, — < — , « > 4 as 

^n\ n\ 2" 

can be easily shown. The series 2_,yI^") is a converging geometric series and by 
Theorem 7.2.1 this implies the convergence of /^(I/m!) as well. 



Example 7.2.2. The series / —p= converges. The proof, similar to that used in 
the previous example, is left for the student. 
Another popular comparison test is given next. 

Theorem 7.2.2 (Limit Comparison Test). Consider two infinite series /a„ 
and 2, ^n such that fl„ > , ii^ > for all n>nQ and let lim — ^ exist and be 
finite. Then, convergence of 2_,^n implies the convergence of /^a„ . 
Proof. 

The relation lim — ^ = M implies — ^ < M + 1 for sufficiently large n> rii. 

Consequently, a„ < (M +1)Z7„ , n > Mj . Since /^^n converges, so is 
/^[(M +l)-i)„] (Theorem 7.1.3) and by virtue of Theorem 7.2.1 so is /a,, ■ 

Corollary 7.2.1. Consider the infinite series /^a„ and /^^„ such that 

b 
a„ > , fo„ > for all n > Mq . If lim — ^ exists and is finite and if 

y b^ diverges, then / fl„ diverges. 

Proof. 

If the opposite holds, i.e., if /^fl„ converges, the series /^^„ must converge by 
virtue of Theorem 7.2.2. However, this contradicts the a priori given divergence 
of this series and therefore 2, ^n must diverge. 
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Example 7.2.3. Let a„ = -j= . The series defined by b^=— diverges and 

n + ^Jn n 



also yields 

. b . ( 

lim — — ^ lim 1 -t- 



lim —5- = lim 



u '^ 



V yi^ J 



which implies the divergence of 7^a„ . 

Our next results is one of the most popular convergence tests. 

Theorem 7.2.3 (Integral Test). Let /(x) , 1 < x < oo denote a nonnegative 
continuous monotone decreasing function such that f(n) = a^ , « > 1 . Then, the 

infinite series ^fl„ converges if and only if the integral /(x)(ix converges. 

1 

Proof. 

Since f(n + 1) < f(x) < f(n) ,n< x<n + \ we may apply Theorem 6.3.3 and get 

rt+l n+\ n+\ 

\ f{n + l)dx< \ f{x)dx< \f{n)dx , n<x<n + \ 

n n n 

which leads to 

«+i 
flj + ^3 + . . . + a„+i < f{x)dx < «[ + flj + • • • + '^n 

n+l 

fl] < f(x)dx 
1 

n+l 

Let /^fl„ converge. Then clearly f(x)dx is bounded (why?) and 

1 

n+l oo 

since f{x)dx also increases with n , the integral f{x)dx converges. On 
1 1 

the other hand, if f(x)dx converges, the inequality 

1 

n+l oo 

^2 +a3 +••■ + «„+! ^ f{x)dx< \f{x)dx implies 



1 

n+l 

Hence S'^+i - a, < | f(x)dx < S„ ,n>l 
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O] < f{x)dx 



■'n+l 

1 



i.e., the sequence {S„} is bounded. Since it is also monotone increasing, it 
converges. This completes the proof. 

Example 7.2.4. Let a„ = — , « > 1 where a is an arbitrary fixed number greater 
n" 

than 1. We associate this sequence with the decreasing nonnegative continuous 

function f(x) = — ,x>l whose integral over [0,oo) exists and equals to 

x" 



C dx 1 1 
J x" ~ \-a 



a-\ 



Therefore, the series > converges for all a>\. 

Example 7.2.5. Since the series ^ — diverges so is the integral — . The 

1 
student must though observe that all the requirements of Theorem 7.2.3 are 
fulfilled. 

Theorem 7.2.4 (Cauchy Condensation Test). Let {«„} denote a nonnegative 
monotone decreasing sequence for n>nQ, i.e., a„ > , a„ > a„+i for all n> Hq. 

Then, ^fl„ converges if and only if ^2"a„ converges. 

n=l n=0 

Proof. 

We may assume riQ =1 (showing that this assumption causes no loss of generality 

n m 

is left as an exercise for the student). Let S„ = /,'^k ^^'l ^m ~ /?■ "^2' denote 

k=\ k=0 

the partial sums of the two series respectively. If « < 2'" we have 

S„ = flj +02 + . . . + a„ < fli + (^2 + ^3 ) + (fl4 + ^5 + flg + a-;) + ... 

(7.2.1) 
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while for n > 2 we get 

S^ = a^+a2+... + a„ > flj + fl2 + (^^3 + 04 ) + («5 + flg + 07 + flg ) + . . . 

a, , r (7.2.2) 

2 + 2^4 + 4ag ■ 



+ (flj-'-i+i + '^2""'+2 + • • • + '^r' ) ^ — + «2 + 2«4 + 4(38 + . . . + 2'" fl 



Both sequences p^},!!",,,} are increasing. Therefore, if {S„} converges it is 
bounded above (Theorem 7.1.2) and since jr,,,} is bounded above by virtue of Eq. 
(7.2.2), it converges as well. Similarly, If {t^} converges, it is bounded above 
which implies (Eq. (7.2.1)) that {S^} is also bounded above and being an 
increasing sequence must also converge. 

Example 7.2.6. We previously proved the convergence of 2, — ,cx>l by 
applying the integral test. Another method is Cauchy condensation test. Indeed, if 
a > the series 2, — whose terms are nonnegative and decreasing, converges 

if and only if the series 7 2" converges. The new series can be rewritten 

as 2,1 — IT which is a geometric series that converges if and only if a - 1 > , 
i.e., a>\. Not surprisingly, the result is identical to that of Example 7.2.4. 



Example 7.2.7. Let fl„ = , m > 2 . The sequence has only nonnegative 

M[ln(M)]" 

terms and is clearly decreasing for arbitrary « > . Therefore, its infinite series 

converges if and only if the series > 2" = > converges. 

^ 2"(ln2")« ^n"(\nl)" 

By Example 7.2.6 this is true for a>\ but not for a = \. Thus, for example, the 

series / does not converge while / r- does. 

^n\n{n) ^^M[ln(«)]^ 

Theorem 7.2.5 (Ratio Comparison Test). Let /^fl„ and /^fc„ be two infinite 
series such that a„ ,Z7„ > and ""*"' < — ^^ for all « > Mq . If / ^« converges then 
2^ cin converges as well. 
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Proof. The inequality ^^ < -^^ yields ^^ < ^^ , m > Mq • Thus, the sequence 

{c„} defined by c,^ = a^/b^ , n>nQ is bounded above by M =a lb . This 
implies fl„ < MZ7„ ,n>nQ. The convergence of /^^„ guarantees the convergence 
of y Mbjj (Theorem 7.1.3) and the comparison test (Theorem 7.2.1) implies the 
convergence of 2, '^n ■ 

1 1 v-' 

Example 7.2.8. Let fl„ = — ,^„= — -By Example 7.1.4 the series y fc„ 

converges. Also 



M + 1 2 



V^ 1 
and therefore > — converges. 






Theorem 7.2.6 (D'Alembert's Ratio Test). Consider an infinite series 
y ^ fl„ such that a^ > , « > Mq • Then 

1. If — ^^ <q ,n>nQ for some ^:0<(j'<l, /a„ converges. 

2. If ""*"' > q ,n>nQ for some 17 > 1 , ^ a„ diverges. 

Proof. 

n+l 

In the first case we have ""*"' <q = — and since /^q" converges, so does 

«n q" 

y a„ , by virtue of Theorem 7.2.5. In the second case we easily obtain 
a„>a„ , M > «o and therefore the general term a^ does not converge to . 
Consequently, by Theorem 7.1.1, ^a„ diverges which completes the proof. 

A direct consequence of D Alembert's ratio test is: 

Theorem 7.2.7 (Cauchy's Ratio Test). Consider an infinite series V'on such 
that a„ > , « > Mq • Then 
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1 . If lim ""*"' < 1 ^ > a^ converges. 

«^~ a„ ^ 

2. If lim — ^^ > 1 => V fl„ divverges. 

The proofs of Theorem 7.2.7 and of the next result are left as an exercise for the 
reader. 

Theorem 7.2.8 (Root Test). Consider an infinite series /^fl„ such that 
a„ > , M > «o • Then 

1 . If !Wa„ < a , M > «2 for some n^ > iiq and < « < 1 , then 2, ^n 
converges. 

2. If !Ha„ >\ ,n>n^ for some n^>nQ, then ^a„ diverges. 

The next result is the classic Cauchy's root test. 

Theorem 7.2.9 (Cauchy's Root Test). Let /^a„ denote an infinite series such 
that a„ > for all n > «q . If 



the series converges and if 



lim ^a„ < 1 (7.2.3) 



lim :^ > 1 (7.2.4) 



the series diverges. 

Proof. 

The fact that several terms in the series (a finite number) may be negative should 
not disturb, since we ignore them and start from Mq on. If the shortened series 

converges, it will still converge if we add the first Mq terms, and if it diverges it 

will diverge with the initial «q terms as well. 

In the first case, find a number q such that lim!Hfl,j < 17 < 1 . There exists a 
positive integer Mj ( ^ Mq ) such that !^fl„ <q ,n>n^. This implies 

a^ < q" , n> fii and by Theorem 7.2.1 and Example 7.1.4 the series converges. 
To show the second part of the theorem we again apply Theorem 7.2.1 and the 
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fact that y^q" diverges for all q >1 (This is left as an exercise for the student). 

If limda^ = 1 the series may converge or diverge and further investigation is 
needed. 

Example 7.2.9. Consider the infinite series 1 + — + 1 + — + 1 + - + ... + 1 + — + .... 

2 4 8 2" 

Here limi^a^ = 1 and it is not clear whether the series converges or not. However, 

all the terms which generate the series are positive and the partial sums are not 
bounded above. Consequently, by virtue of Theorem 7.1.2, the series diverges. 

Example 7.2.10. The infinite series — H — r- + — ^H — - + ... provides 

3 5' 3^ 5' 

lim!^fl„ = — < 1 , i.e., it converges. 



PROBLEMS 

1 -A 

1 . Let a„ = r= , M > 1 . Show that > a^ diverges. 

0.1« + 5Vn i~^ 

-A 1 

2. Use the integral test to show that > — 1= converges. 

A 1 v^ 1 

3. Use the integral test to show that > diverges and > r- 

t^«ln(«) tin(ln(n)f 

converges. 

-A 1 

4. Does the series > , a > 1 converge? 

°° VlY f2^" 

n=Q 



+ 

2 13 



5. Use Cauchy's ratio test to show the convergence of 2, 

«=o 

-A 1 

6. Can you use Cauchy's root test to show that /^—r is convergent. 

7.3 Conditional and Absolute Convergence 

So far we have mainly discussed infinite series with nonnegative elements or with 
almost only nonnegative elements, i.e., /_^a„ such that a„ > for all n>nQ. 
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However, quite often this is not the case. For example, a very popular infinite 
series is an alternating series defined as follows. 

Definition 7.3.1. Consider a sequence {a^} whose terms are all nonnegative. The 
series ^(-1)"""" a^ , ^(-l)"a^ are called are called alternating series. 

. , , . ^, .,111 ,1111 

Example 7.3.1. The series 1 1 h... and 1 1 ... are 

234 3579 

,.,,11111 

alternating series while iH 1 1 ... is not. 

2 3 4 5 6 

If by removing the first n^ terms of a series, an alternating series is obtained, the 
original series can be treated as an alternating series for all convergence matters. 

Theorem 7.3.1. (Leibniz's Alternating Series Test). Consider a sequence {a^} 
whose terms satisfy < a„+i < fl„ , « > Mq ^rid lim a„ = . Then, the series 

J](-l)"^'a„ ,J^(-ira,^ converge. 
Proof. 

(a) /^(-l)"^ a„ ■ Without loss of generality we may assume n^ =1 . The series 
is therefore 

ai - a2 + a^ - a^ + a^ - . . . 

and it clearly satisfies 

S2 <S^<Sf, <-<Ss <S^<Si 

The sequence {S2„} is increasing and bounded above by S^ . Consequently, it 

converges to some S . Also, the sequence {52„_i} is decreasing and bounded 

below. It therefore converges to some S . However, 52„+i - S2„ = a2«+i which 
yields 

lim "^2«+i - lim S2„ = lim Ojn+i = 

n— >co tt— >oo tt— >oo 

Thus S = S "^ , i.e., {5„} converges as stated. 
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(b) ^(-l)"a„ . The series is -a^ + a2 - a^ + a^ - a^ +... , it satisfies 

5'i<5'3 <S^ <---<Sf, <S^<S2 
and again, the odd and even partial sums must converge to the same value. 

Example 7.3.2. The series 1 1 \- ... converges since the sequence < — 

2 3 4 [n 

fulfills all the requirements of Leibniz's test. On the other hand, the sequence 

— , — A, — , — , — , — ,—,—,...} satisfies lima„=0 but is not a decreasing 
32654987J «^~" 

sequence. The problem of determining whether the associated alternating series 
converges is left as an exercise for the student. 

.,,-,^ •. , f, 11111111111 1, 

Example 7.3.3. Consider the sequence <1,— , — , — , — , — , — ,—,—,—,— , — ,...>. It 

[23434565656J 

satisfies the requirement lim fl„ = , it is not a decreasing sequence and its 

associated alternating series does not converge. This can be shown as follows. A 
subsequence of the generated alternating sequence is 

c ill ^12 ^12 3 

5, =1 = - , S(,= — + , 5,,=- + + , ... 

2 2 23-4 23-45-6 

which can be rewritten as 

^2 ^ 2 [ 3J 2V35 

These partial sums are not bounded and this yields that the series diverges. 

Definition 7.3.2. An infinite series ^a„ is absolutely convergent (or converges 
absolutely) if the. series /^|«„| converges. 

Example 7.3.4. The series 1 r- + ^r + ... converges. Indeed, by Example 

22 32 42 

7.2.4, the series 2, — converges for all a>\, and in particular for a = \. 
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sin(M) 



Example 7.3.5. The series 2, j — is absolutely convergent. Indeed, 

n 

<^- and /^—^- What about 2, j — which is not an alternating 



sin(n) 



series 



°fE 



n — n 

sin(n) 



n 



? The next fundamental result provides the answer. 



Theorem 7.3.2. A series which converges absolutely is also convergent, i.e., the 
convergence of /^|a„| yields the convergence of /^a„ as well. 



Proof. 



If 7^|a„| converges, then, for an arbitrary £ >0 there exists an integer Mq > 
such that I a„ I + 1 a„+i I + • • • + 1 «m I < ^ for all n^ <n< m . But since 



\a„ +a„., +... + a,„\<\a„\ + \a„.,\ + ... + \aJ<£ 



m\-\"n\^\ "«+l 



The partial sums of 7^fl„ form a Cauchy sequence and the series converges. 

The opposite is clearly false. For example, the harmonic series diverges 
(Example 7.1.2) while its alternating series converges by Leibniz' test. Also, if 

y a„ diverges, so is ^ J a„| , otherwise this contradicts Theorem 7.3.2. 

Example 7.3.6. Let /.'^n ~ z_j ^ — • This infinite series is not a monotone 

n 

decreasing alternating series and Leibniz's test for convergence cannot be applied. 



Yet, 



cos(«) 



<^- which, by virtue of Theorem 7.2.1 (Comparison test) and 
n 



cos(m) 



Hence, by Theorem 7.3.2, 



Example 7.2.4, implies the convergence of ^ 

Zcos(m) 
— converges. 
n 

Consider now a general infinite series /^a„ and define the sequences 
|a^|, ja^l as follows: 



"" 0,«„<0 ' "" l0,a„>0 



(7.3.1) 
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Thus, jfl^l consists of all the nonnegative terms with O's replacing all the 

negative terms and ja^ | - of all the negative and zero terms with 0' s replacing all 
the positive terms. Clearly, 

««=«;!"+< '!««! = «« -«« (7-3.2) 

for all n . Let 5^ , 5^ , S^ denote the partial sums of the associated infinite series 

of jfl^j, jfl^j, |a„|| respectively. Then, Eq. (7.3.2) yields S^=S'^+S'^ and 

5„ = S'^ - S'^ . The relation between the convergence of S and that of S'^ , 5^ 
is given by the next result. 

Theorem 7.3.3. The series /^|fl„| converges if and only if both sequences 
P«^P« I converge. 

Proof. 

(a) If both |5^|,|5„| converge, then lim 5„ = lim 5^ - lim 5~ , i.e., /^|«„| 

converges. 

(b) If ^^\ converges then the comparison test (Theorem 7.2.1) implies the 

convergence of p^j , {-■S„| and the convergence of |-5',7| guarantees the 
convergence of p"). 

Corollary 7.3.1. If one of the sequences p,!), p,7| converges while the other one 
diverges, then both /^a„ and /^|fl„| diverge. If both p^j, p„| diverge, 
/^l fl„| diverges, while /^a„ may or may not converge. 

Proof. 

Let for example p,^| converge while p^j diverges. By Theorem 7.3.3 the series 
^1 a„| must diverge, since its convergence would imply the convergence of both 
p^l and p7j. Also, /.'^n must diverge since its convergence, the convergence 

of p^ I and the relation S„ = 5„ - 5^ , implies the convergence of p ~ j - which 
was assumed to diverge. The proof of the second part is left as an exercise for the 
student. 
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Definition 7.3.3. The series ^«„ is conditionally convergent if y fl„ 
converges while /^|a„| diverges. 

Example 7.3.7. The series 1 1 \-... is conditionally convergent while 

, 111 ...,,, 

the series 1 ^ + — ; t + --- is not since it is absolutely convergent. 

22 32 42 

Example 7.3.8. Let ^a„=2-- + + + .... Here both j^,! | , j^,; | 

diverge and {5'„} diverges as well, i.e., /.'^n i^ ^"'^ conditionally convergent. In 

the case of the series 1 1 h... the sequences p^j,p„| diverge, but 

{^jj} converges. 

Theorem 7.3.4. If /^a„ is conditionally convergent, both sequences p,!), p^j 
must diverge. 

Proof. 

Let p,^] converge. The relation 5,7 = S„ - S'^ implies the convergence of p^j 
and by virtue of Theorem 7.3.3 /^|fl„| converges. Thus /^a„ is not 
conditionally convergent but rather absolutely convergent. 

Definition 7.3.4. Let /^fl„ be an infinite series and let the relation I = l(k) 
denote a one-to-one correspondence from the set of all positive integers 
onto itself. The series /^^t where bi^ =«/(*) for all k , is called a rearrangement 

of ^a„ . 

Example 7.3.9. The series In 1 1 \- ... is a rearrangement 

3245768 

of the series 1 1 1 1 ... . The relation / = l(k) is 

2 3 4 5 6 7 8 

/(4; + l) = 47+l,Z(4y + 2) = 4; + 3,/(4y + 3) = 4y + 2,/(4; + 4) = 4y + 4 

for an arbitrary integer j >0 ■ 

Two fundamental results related to rearranged series are given next. 
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Theorem 7.3.5. If /^a„ is absolutely convergent and /_^bi. is a rearrangement 
of 2, '^n ' '^hen 2, ^k converges to 2, ^n ■ 

Proof. 

Clearly /^a„ converges since the series is absolutely convergent. Let us 
distinguish between two cases. 



(a) a„ > for all « > 1 . Consider an arbitrary partial sum of /.^k ' ^^Y' z^^k ■ 

k=i 

Since all the terms of /^«„ are nonnegative, we clearly have 

m m N 

2^bi^ =2,cii(k) -/.'^n foi" ^11 A' > max{/(l),/(2),...,/(m)}. Consequently 

k=l k=l n=l 

the rearrangement converges and /^^^ ^ y,'^" ■ Since /^a„ is also a 
rearrangement of /^^^ we get /^fl„ ^ /.^t ^^ well which yields 

(b) The terms a^^ are not necessarily nonnegative. In this case we generate the 
sequences |fl^|,|a„| and pt^j,p<^j- Consider a partial sum /^^^ of the 

k=l 

rearranged series. Then 

m m m 

Y.^k=Y.K+Y.^~k (7.3.3) 

k=\ k=\ k=\ 

The series /^fl^ is clearly absolutely convergent and its terms are all 
nonnegative. Therefore, by the proof of part (a) /Pt i^ absolutely convergent 
and converges to /^«,| ■ Similarly, -^.cin i^ absolutely convergent and its 
terms all nonnegative, which yields the absolute convergence of - /PI '■'-' 
-^ a~ . Consequently, /P^ converges to /.'^n and by virtue of Eq. (7.3.3), 
/ b), converges to /^a„ ■ 
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Theorem 7.3.6. If each arbitrary rearrangement of a series 2, ^n i^ convergent, 
then /.'^n is absolutely convergent. 

Proof. 

First, since /^fl„ converges in its original arrangement, we must have 

lim a^ = lim a^ = lim a^ = . If 2, '^n i^ '^^^ absolutely convergent then, by 

virtue of Theorem 7.3.3 one of series /^fl^ , /^fl„ does not converge. For 

example, let /,'^n diverge and we will generate a rearrangement that diverges, 
in contradiction to the given information. Since zeros in the original series appear 
in both ^fl^ and /.'^n we will rearrange a// the elements of /^a^ and only 

the negative ones in /.'^n ■ Define an increasing sequence of positive integers 

{«! , M2 > • • • > ''i ' • • •} such that 

flj + 02 + ... + Ojj + a'j >1 



<_,+i+<_,+2+--- + <+«A>l 



where a- is the ^ -th negative term in |a„j. This is possible since /^fl^ , 
composed of nonnegative elements, diverges, and since lim a^^ =0 . Now, 

rearrange the series 2, ^n ^^ 

+ <_,+! +<_,+2 +••• + < +«A+- 

and denote its partial sums by T,, . Then r„ ^.j > 1 , r„ ^.2 > 2 , . . . , r„ ^.^ >k ,... 
which implies that the sequence {r^j diverges. Thus, both series /.'^n ^^d 
y a^ converge, i.e., /.'^n is absolutely convergent as stated. 



7.3 Conditional and Absolute Convergence 201 

n 

Lemma 7.3.1. Let Y a„ , /^^„ denote two arbitrary series and let S^ = /^a„ • 

1=1 
Then 

n 

Y,akh =S,(b,-b2) + S2ib2-b,) + ... + S„_,(b„_,-b„) + S„b„ (7.3.4) 

k=l 



Proof. 

The proof is by induction. For « = 1 both sides of Eq. (7.3.4) equal to afy 
(5] =0] ). Assume the lemma to hold for some « > . Then 

«+l n tt-1 



k=l k=l k=l 

n-\ 



Yj^'kh =^a^b^ +a„+i^„+i =X^'t(^* -bM) + S,J}n +«„+A 

k=\ k=\ 

n-\ 

n 
/ , ^k (i'k ~ i'k+l ) +"5/1+1 ^n+1 



ylr=l 



i.e., Eq. (7.3.4) holds for n + l as well. 

Lemma 7.3.1 provides another useful convergence test. 

Theorem 7.3.7 (Dirichlet's test). If a series V' a„ has bounded partial sums and 
if {b„} is a decreasing sequence which converges to , then y.'^n^n converges. 

Proof. 

n 

Let I ^nl < M , « > 1 where S„=y^ai^ .By Cauchy criterion we must show that 

given £ >0 an integer «q > can be found such that 

I a„+i^„+i + fl„+2^„+2 + • • • + flm^m I < ^ fo^ ^^ m > 11 > Hq . By Lemma 7.3.1, since 
the terms of p,,} are all nonnegative and bj^ > bj^^^ , we get 
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a„+i^„+i + a„+2^«+2 + • • • + a,„b„ 



in— I 

X ^k (h - bk+i ) + S,„b„ - S„b„ 



k-n 
m— 1 



k-n 

<Mib„-bJ + Mb,„+Mb„=2Mb„ 
Since lim Z?^ = an Mq > exists such that i>„ < , n> iiq and this yields 

n^~ 2M 

I a„+i^„+l + «„+2^«+2 + • • • + «m^m | < £• , OT > « > «o 

which completes the proof. 

Example 7.3.10. A well known result from trigonometry is the identity 

. ( nx\ . ( (n + \)x 

sin(x) + sin(2x) + . . . + sin(Mx) = — ^ — - — y^ — (7.3.5) 

sin — 

which holds for all x ^ . The right-hand side of Eq. (7.3.5) is bounded in 
absolute value by (I sin(x/2|j for all n . Since the harmonic series satisfies the 

requirements imposed on p^l in Theorem 7.3.7, the series > converges 

tt-i 
for all X ^ while it converges to for x = . 

The requirement that {b„} is a decreasing sequence which converges to , is 

quite fundamental in showing the convergence of 2_,^nbn ■ ^^ '^he next example 
both /^a„ and / Z?„ converge, yet y.'^nbn diverges. 

ZV 111 
a„ = > fe„ =1 — j= + —i= — r= + Both series 
•v2 •v3 '\/4 

Zl 1 1 
a„b„ =l-\ h — + — + ... diverges. 

Theorem 7.3.8 (Abel's test). Let /"'a,, converge and let {b^} be a bounded 
monotone sequence. Then 2,'^nbn converges. 
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Proof. 

Clearly the sequence {Z?^} converges to some B and without loss of generality we 
may assume that it is increasing. Consider the sequence \B-b,^\ which is 
decreasing and converges to . Since /^fl„ converges, its partial sums must be 
bounded and by Theorem 7.3.7 the series /^a„(Z?-i>„) converges. Clearly, the 
series 2,^n^ converges as well and the relation 

'^a„b„ =Y^a„B-^a„iB-b„) 

guarantees the convergence of 2_,^n^n ^^ ^ difference between two converging 

series. 

PROBLEMS 

, o. . , • 1 1 1 

1. Show that the series — 1= 1=-\ ;=-••• converges. 

2V2 3V3 4V4 

^ o, , , • 1 1 1 

2. Show that the series 1 ... converges. 

21n(2) 31n(3) 41n(4) 

3. Which of the following is absolutely convergent series: 

(a) 1 + + . . . (b) sin(l) - sin — + sin - - sin — + . . . 

21n(2) 31n(3) 41n(4) UJ UJ U6J 

, ^ ., , ,1111111 

4. Consider the series 1 1 1 1 h... and its 

2 4 8 16 32 64 128 

,1111111 , , , 

rearrangement 1 1 1 1 \- ... (a) Are they 

8 2 4 16 128 32 64 

convergent? (b) Do they converge to the same value? 

5. By theorem 7.3.6 there must be a rearrangement of the series 

1 1 \- ... that diverges (why?). Find one. 

2 3 4 

7.4 Multiplication of Series and Infinite Products 

In this section we will denote an arbitrary series by 2, '^n instead of 2, '^n ■ The 
reason is the association made here between 2, '^n ^^d the power series (treated 

n=0 
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in detail in Section 7.5) ^a„x" =aQ +fl]X + a2-^ +... + a„x" +... . Consider 

two formal power series ^a^x" and 2,l^n^" ■ ^^ v^e formally multiply these 

expressions and for each power x" denote the sum of all the coefficients which 
multiply this power by c„ , we get 



n 

k=Q 



(7.4.1) 



Thus, we may formally write 



V/=o 



, ,=0 J 



n=0 



(7.4.2) 



where the coefficients c„ are determined by Eq. (7.4.1). If we now substitute 
X = 1 we obtain the formal relation 



A 



T.^'n X^« =X^« 



(7.4.3) 



which provides the motivation for the following definition. 



Definition 7.4.1. The Cauchy product of ^a„ and ^^„ , is the series ^c,, , 
where 



«=o 



Cn =X'^*^«-* '"-^ 



(7.4.4) 



k=0 



Notice that nothing related to the convergence of the two given series, was 
assumed. 
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Zl 1 1 v 111 

a„ =1 + — + — + - + ... and > b„=\ + - + — + — + .... 
2 4 8 ^ 3 9 27 

Then 

^ ^ ^ ,11,2 1 1 1 1 19 

Cn =1-1 = 1, c, =!•- + — -1 = — ,c, =!• — + + — -1 = — ,... 

" 32 39234 36 

Theorem 7.4.1. If ^a„ and /.^n ^'"^ absolutely convergent, their Cauchy 
product /.'^n i^ ^1^0 absolutely convergent and Eq. (7.4.3) holds. 

Proof. 

Let A^,B^,C,^ denote the partial sums of /.'^n ' zPn ' z.'-n respectively. We 
distinguish between two cases: 

(a) a„ > , ii„ > for all « > . Consider the product 

It certainly contains all the terms a,i> , i + j <n , which implies C„ < A^B^ . On 

the other hand each term of the product is of the form a,i> , i + j <2n which 

yields A^B^ -Cin ■ The sequence {C„} is thus increasing and bounded hy AB 
where 

A = lim A„ ,B= lim B„ 



Consequently, ^ c,j converges to some C=limC„ where C<AB. Since all 

c,j are nonnegative, /^c^ is also absolutely convergent. Finally, by virtue of the 
relations 

limC„ =]mC2« =C , A„B„<C2„ 

we get AB<C,i.e., C = AB. 

The general case. Since the two series are absolutely convergent, /^|fl„| and 
^1 Z?^ I converge. Denote the Cauchy product of these series by /.^n ■ Then 
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n 

Co=|ao||^o|'Ci=|flo||^i| + |ai||^o|'---'C„=^|ai||Vi|'--- 

A:=0 

and by the proof of part (a) the series 

/ C„ = I flg 1 1 ^0 1 + 1 "^0 1 1 ^1 1 + 1 '^1 1 1 ^0 1 + 1 "^0 1 1 ^2 1 + 1 "^1 1 1 ^1 1 + 1 ^^2 1 1 ^0 1 + • • • (7.4.5) 

converges. Notice that since all the terms of each c,j are nonnegative we do not 

need any pair of parenthesis at the right-hand side of Eq. (7.4.5). Hence, the series 
defined as 

y c'^ =aQbQ +ao^i +fli^o +^^0^2 +«i^i +fl2^o +••• 

converges as well (cq = flo^o ' ^i ~ "^o^i ' '^2 ~ '^i^o '•••)• Notice that this series is 
not 2, c„ which was defined as 



^c„ =flo^o+(«o^i+«i^o) + ---+ X"^*^"-' 



k 
\k=Q J 



+ ... (7.4.6) 



but rather obtained by removing all pairs of parenthesis from the right-hand side 
of Eq. (7.4.6). Since /.'^'n converges absolutely, any rearrangement of its terms 

yields a series with identical sum. Also, the partial sums of /^c„ are also partial 

sums of /^c'j and therefore /^c^ converges and equals to /^c,' . We now 

perform a rearrangement of 2, c« such that for each n the first (« + 1) terms 

will be those obtained by the expansion of 

A,jZ?,j = (aq + fl[ + ... + a^ )(bQ + i>[ + ... + b^) . This is feasible since all the terms in 

A„B„ appear also in A„^.iZ?„^.i . The sum of the rearranged series equals to 
y c^ but since A„B„ are a subset of its partial sums we also get 

X'^« =!!<' = 1™ ^nB„ = lim A„ • lim B„ = (^aJ^Z^J (7.4.7) 

Thus, the Cauchy product of the two series converges to the product of their sums. 
The absolute convergence of /^ c„ follows from the inequality 

kJ^-^n =|fl0^J + |«lVl| + --- + |«A| 
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combined with the comparison test (Theorem 7.2.1) applied to the series 2_,\ c«| 
and 2, dn ■ This completes the proof. 

Zv 111 

fl„ =>Z7„ =1 + — + — + - + ... = 2 . Both series are 

absolutely convergent. Hence their Cauchy product converges and equals to 4 . In 
particular we obtain the identity 

n + \ 



n=0 ^ 



as may be shown by the student. 



The next results present additional relations between 7^a„ , 7^^„ and their 
Cauchy product 2, c« • 

Theorem 7.4.2 (Merten's theorem). If /_'a„ , /_'^„ converge to A,B 

n=a «=0 

respectively and one of the series, say /.'^n ' i^ absolutely convergent, then 

n=0 

Cauchy product of the two series converge lo AB . 
Proof. 

n n 

Let A^ = 2,'^k ' ^n ~ zPk denote arbitrary partial sums of two series. The n -th 

k=Q k=0 

partial sum of the Cauchy product is 

C„ = Co + Ci + . . . + c„ = flo^o + (flo^i + fli^o) + • • • + («o^! + ai^«-i + • • • + ««^o) 

= «0'B„+fliB„-i+... + a„5o (7.4.8) 

and by denoting P^= B^- B we obtain 

C„=a^,{B + p„) + a,{B + p,) + ... + a„{B + p,,) 

= A„B + «o A + «i^„-i +--. + aJo= A„B + r„ (7.4.9) 

Since lim A„B = AB , it is sufficient to show 
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lim r„ = lim {aaPn + flj Ai-i + • • ■ + «„;^o ) = 

The convergence of ^ b^ yields lim J3„=0 . Consequently, {y5„ } is bounded, 

i.e., |/?„| < ^ , « > for some finite K . Given an arbitrary e >0 , choose Mj > 
such that 

\Bj<-L,n>n, (7.4.10) 

where A = ^ I '3„ I • Next, let Mj > be such that 

n=0 

III I 11^ 

\an\ + \an+i\ + --- + \a,„\<—-,m>n>n2 (7.4.11) 

The feasibility of both choices follows from the convergence and absolute 
convergence of /.^n ^^'^ z.'^n respectively. Now, let n> 11^+112 -\ ■ Then 

«=0 n=0 

|''«|=|«0>9«+«lA-l+--- + «,A|^|«o||A!| + |«l||A!-l| + --- + |««-„,||>^«,| 

+|««-«,+i||>^«,-i|+|««-„,+2||A!,-2| +••■+! ««||>^o| (7.4.12) 

The first term of the right-hand side of Eq. (7.4.12) satisfies 

|«o||^«|+|«i||^«-i|+---+|«„-«,||A.J<(«o|+|«i|+---+|««-„,|)^^| (7.4.13) 

while the second term, since « - Mj + 1 > «2 (why?) provides 

I ««-«,+! II >^«,-l I + I ««-„,+2 II A,,-2 1 + • • • + 1 «« II >^0 1 < I ««-,,,+! I + I ««-,,,+2 1 + • • • + I a« IJ- ''^ 

<—K = - (7.4.14) 

2K 2 

Therefore, | /;J < £■ for « > «; + «2 ~ 1 and the proof is completed. 
The next result is given without proof. 
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Theorem 7.4.3 (Abel's theorem). If /_'«„, zl^n converge to A,B and 

«=o «=o 

Cauchy product of these series converge to C , then C = AB . 

The last topic in this section is infinite products defined as follows. 
Definition 7.4.1. For an arbitrary sequence {a„} denote 



A=n«i=«i ; ^„=n«*= n 



k=l k=l 



a„,n>2 (7.4.15) 



\k=i J 



The sequence {P„} is a sequence of parria/proc/Mcfa which converges to I I flit if 

k=l 

and only if lim P„ = P for some finite P . In this case we write I I fl^^ = P . If 

k=l 

{P„} converges then the infinite product I I aj; converges, otherwise it diverges. 

k=\ 

Example 7.4.3. Let a„ = ,n>\. Here P„ = TTga: = > i-^' 1™ K = 

n + 1 -*■-*■ n + \ «^~ 

which yields I I fl^^ = . 



k=l 



k=\ 



Theorem 7.4.4. If M Uj^ converges to P ^^ then lim a„ = 1 . 



Proof. 



/t=i 



P limP„ p 



Since a„ = — ^^ we get lim a„ = "^°° = — = 1 . 

P„_i «-- limP„_i P 



Theorem 7.4.5. If a„ > for all n then I I a,;, converges if and only if 
^ln(a^). 



k=\ 



k=\ 
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The proof is left as an exercise for the student. 

PROBLEMS 

1 . Find the Cauchy product of l-\ 1 1 1-... and 1 H 1 1 \- ... 

2 4 8 3 9 27 



111 ,,111 

- + - + - + ... andl + — r- + -^ + — r 

2 4 8 2^ 3^ 4^ 

1 1 1 



2. Find the Cauchy product of 1 + — + — + — + ... and 1 H — r- + —^-\ — ^ + . . . 



3 . Let 2, ^n- /, ^n =1 — /= + ~i= — /= + • • • (^) Fiii'l the Cauchy product of the 

■v2 ■\j3 ^4 

given series, (b) Is the Cauchy product convergent? 



7.5 Power Series and Taylor Series 

Definition 7.5.1. An infinite series 



^a„(x — fl)" =aQ +aj(x — fl) + ... + a^{x — a)" + ... (7.5.1) 

where the coefficients fl„,n > and a are real numbers and -o° < x < oo is called 
a power series about a . 

Example 7.5.1. The series x + x + x +... + X """"'+... is a power series about . 
All the odd coefficients are zero, while the even ones equal to 1. 

The next result relates to the convergence of a power series. For sake of simplicity 
we assume a = . 

Theorem 7.5.1. Assume that a power series converges for some fixed x = Xq ^ . 
Then, it converges absolutely for arbitrary x < Xq and converges uniformly for 
x< R for any fixed R < Xq . 

Proof. 



Since /^a„Xo converges, its general term converges to 0. Consequently, an 
integer A' such that fl„Xo <1 ,n> N exists. Hence 
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«=0 



«=0 



(7.5.2) 



Since \x/xq\ < 1 and by virtue of fl„Xo < 1 , m > A'^ , the right-hand side of Eq. 

(7.5.2) converges. The second part of Theorem 7.4.1 is now easily concluded and 
is left as an exercise for the reader. 

Example 7.5.2. Consider the power series 



X X 

x + — + ... + — + ... 

2! n\ 

It is easily seen to converge for x = l. Consequently it converges absolutely for all 
X < 1 and converges uniformly for x< R where R is an arbitrary number less 
thanl. 

Example 7.5.3. The power series 



y nx" = x + 2x + . . . + nx" + ... 



(7.5.3) 



converges for all arbitrary x , x < 1 . Indeed, let a„ denote the general term of the 
series and Ixl < 1 . Then 



*n+\ 



(m + 1)x 



(7.5.4) 



provided that (why?) 



n > 



(7.5.5) 



Consequently, since jlxl < 1 , the power series converges absolutely and given an 

arbitrary fixed R <1, converges uniformly for a\lx< R . 

If a power series diverges for some fixed x = Xg ^^ , then it must diverge for 

all X such that |x|>|xq|. Indeed, if it converges for such x, then, by 
implementing Theorem 7.4.1 it must converge for Xg as well! We thus get the 
following result. 
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Theorem 7.5.2. For an arbitrary power series ^a^x" there are two possible 

n=0 

cases: 



1 . y^ a„x" converges for all x ,-^ < x<^ . 



n=0 

2. A finite R>0 exists such that ^fl„x" converges for Ixl < R and diverges for 

«=o 
Ixl > /? . The number R is called the radius of convergence. 

The next result provides a procedure for calculating the radius of convergence. 

Theorem 7.5.3. Given an arbitrary power series ^a„x" , its radius of 

«=o 
convergence R is obtained by 

— =lim«/jflj (7.5.6) 

Proof. 

Let K = lim !?/|fl„ I . Then, for arbitrary fixed x we have 



limn a„x" =|jc|/^ (7.5.7) 

and consequently, by Cauchy's root test (Theorem 7.2.9) it implies that ^a^x" 

n=0 

converges for Ixl ^ < 1 and diverges if Ixl ^ > 1 . Thus, if ^ > the radius of 
convergence is 

R = ^ 
K 

while in the case ^ = the series converges for all x , - 1» < x < oo . 
Example 7.5.4. Consider the power series 
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3 2n+l 

l + x + 2x^ + — + 4x* +... + — + 2nx^" +... (7.5.8) 

3 (2« + l) 

The reader can easily obtain 

lim !?/|a„ I = 1 

which implies that the series is convergent for \x\<l and divergent for |x|>l. 
Clearly, the series is not convergent for 1x1= 1 (why?). 

The next theorem, given without proof, relates to the derivative of a power series. 

Theorem 7.5.4. Consider a power series ^a^x" with radius of convergence R . 

Then, the function f(x) = ^a^x" , Ixl < /? possesses a derivative at this domain 
which satisfies 

fXx) = ^na„x"-\\x\<R (7.5.9) 

Moreover, /(x) is infinitely differentiable at Ixl < R and as in Eq. (7.5.9) the 

various derivatives are obtained by repeatedly differentiating the power series 
term by term, i.e., 

/(*)(x)=^M(«-l)---(M-;fe + l)x""* ,\x\<R (7.5.10) 

Example 7.5.5. Let f(x) = x + 2x^+3x^+... + nx"+.... Its radius of 
convergence is 1 . Hence 

/'(x) = l + 4x + 9x^+... + M^x""'+...,|x|<l 
/'(x) = 4 + 18x + 48x^+... + «2(«-l)x""^+...,|x|<l 

etc. 

We end this chapter by introducing the power series somewhat differently, using 
the concept of Taylor series. 
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Taylor series 

Let a function f(x) and its derivatives f'(x),f"(x),...,f"'(x)he defined and 
continuous in a closed interval [a,b] with /'""""'^(x) continuous at the open 
interval {a,b) . By Taylor's theorem (see Section 5.4) 

f(x) = f(a) + f'(a)(x-a) + ^-^(x-af+... + ^-^(x-ar+R„ (7.5.11) 
where the remainder 

R=^ ^-^ix-a)"^\a<^<x (7.5.12) 

(m + 1)! 

is called remainder by Lagrange. Another form for the remainder 

R„=^ ^■^^{x-^')"(x-a),a<^'<x (7.5.13) 

n\ 

is called remainder by Cauchy, and usually ^ ^ ^' . Clearly, if n varies, ^ 
usually varies as well. If lim /?„ =0, then f(x) can be expressed by a power 



f{x) = f(a) + f'{a){x-a) + ^^(x-af+... + L^(x-a)"+... (7.5.14) 

2! n\ 

This series is called Taylor series of f{x) about a . The existence of a formal 
Taylor series (as a converging series!) of a function is no guarantee that 
Eq. (7.5.14) holds. In addition to its existence, the condition lim R„ =0 must hold 

as well. 

The advantage of representing a function by its Taylor series is obvious. An 
infinite power series as are polynomials, is easy to manipulate and study its basic 
properties and behavior. 

Example 7.5.6. Let f{x) = sin(;t) , - oo < x < oo . The remainder by Lagrange is 

|x"^'a(^)| 
\rJ 



(n+iy. 

where A(^) is either sin(,^) or cos(,^) , i.e., \A(^)\ < 1 . Since, for all real x 
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lim — = 



we get lim R„ =0 and consequently 

3 5 2n-l 

sm(x) = x- — + - — ... + (-1)""'-^ + ... , -oo<x<oo (7.5.15) 

3! 5! (2«-l)! 

Similarly we obtain 

2 4 2n-2 

cos(x) = l- — + — -... + (-1)""'-^ + ... , -oo<x<oo (7.5.16) 

2! 4! (2«-2)! 

Example 7.5.7. Consider the exponential function /(x) = e^ , -oo < x < oo . 
Since f'(x) = e" (see Corollary 5.3.2) we easily get 

/(«)(x) = e\„>l 

and the function's Taylor series is 

x^ x" 

\ + x + — + ... + — + ... 
2! n\ 

The remainder by Lagrange 

«+i 

X 



R„=— e^ ,0<<^<x 

" (n+iy. 

converges to for arbitrary fixed x and thus 

2 n 

X X 

e"" =\ + x + — + ... + — + ...-oo<x<oo (7.5.17) 

2! n! 

A case of a function y = f{x) whose Taylor series exists and converges - but not 
to fix) is given next. 



Example 7.5.8. Consider the function 



y=r ' '""^^ (7.5.18) 

,x = 
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It is infinitely differentiable for all x ^ and by using induction we get 

y<">=e-'/"V„f-lx^O (7.5.19) 



where P„(m) is a polynomial in M . By Eq. (7.5.17) we get e^ >x^jn\ for all 

e 
X > and consequently can easily derive lim = for all positive integer 

m , which, combined with Eq. (7.5.19) leads to 

lim(l/x));<"'(x) = 0,«>l (7.5.20) 

x->0 

Finally, by using Eq. (7.5.20) and induction, we get y "' = , « > . The Taylor 
series in this case is thus identically zero, while the function itself is not. This 
implies that Lagrange remainder of the function for arbitrary X does not converge 
to zero as « ^ oo . 



PROBLEMS 

2 3 

X X 

1 . Let f{x) = \ + x-\ 1 h . . . . Find the radius of convergence, and the first 

two derivatives. 

2. Find Taylor series of Inll + xl about x = . Find the radius of convergence and 
show that the remainder approaches zero. 

3. (a) Repeat problem 2 for /(x) = ln|l-x| . (b) What can be said about 

1 + x ^ 
1-x ■ 

4. Obtain the derivative of sin(x) directly and by using its Taylor series. 

5. Obtain the Taylor series tan" (x) about x = and find its radius of 
convergence. 

6. Obtain the Taylor series of e^ about x = and show that it converges 
uniformly for Ixl < 100 . Does it converge uniformly for Ixl < 200 ? Explain. 



y = \n 



8 Fourier Series 



In the previous chapter we have studied how functions which satisfy certain 
requirements can be replaced by an infinite series of powers of X . The main 
advantage of this fact is that a power series or a polynomial, representing a 
function, is easily manipulated for various applications. 

There are however other ways to decompose a function into simple elements. 
The most popular is given by the Fourier Series Theory, named after the French 
mathematician Joseph Fourier (1768 - 1830), who, in 1822, started to investigate 
this theory. The first main result is that under certain conditions, an arbitrary 
periodic function /(jc) , can be replaced by a trigonometric series, which includes 

sine and cosine functions. This representation is very useful in mathematics, 
physics and engineering sciences. 

We open this chapter with basics of Fourier Series Theory. 

8.1 Trigonometric Series 

Definition 8.1.1. Consider an integrable function f(x) over the interval [a,b\ and 
let wix) denote a weight function that satisfies w(x) >0 , a< x<b . The number 






2 



jwix)[f(x)f 



_ dx 



(8.1.1) 



is called the norm of f(x) over [a,b\ with respect to w(x) . It has the following 
properties: 

(a) 11/ + gll < ||/|| + ||g|| for arbitrary f{x) and g(x). 

(b) ||c/|| = |c|||/|| for arbitrary function f(x) and constant c. 

(c) If ||/|| = and f{x) is continuous, then f(x) =0 ,a< x<b . 
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1 e^-1 

Example 8.1.1. Let w(x) = 1 ,a = ,b = l. Then \\x^\\ = —/= , \\e^\\ = J and 

II II VJ II II V 2 

II 2ll 1 

\\x -X \\ = — ^ . The proof is left for the reader. 

" " 2V3 

The norm defined by Eq. (8.1.1) can be used for determining how close two 
functions are to each other. We call 11/ -^11 the cfwtance between f(x) and g(x) 
and say that a sequence of functions {/„ (x)} converges to f(x) in norm, if 

lim||/„-/|| = 
We next introduce the formal expression 

1 / J777V I 

(8.1.2) 



2 



^ + g^«„cos^— J + Z,„sm^— J 



which is called a trigonometric series. At this point, nothing is said about whether 
the series converges. 

One of the most important features of trigonometric functions is their 
periodicity. 

Definition 8.1.2. A function /(x) , xe D is called periodic if 

fix + T) = f(x),xeD (8.1.3) 

for some constant T . The number T is called a period of f(x) . Every integer 
multiple of T is also a period. 

We will always assume T >0 . Otherwise, reverse the direction of the x- axis. 
The smallest T which is still a period of f{x) , is called the fundamental period 
of f{x) . If two functions f{x) and g(x) share the same period T , then T is 
also a period of f{x) + g(x) . The next result presents some of the properties of 
the trigonometric functions in Eq. (8.1.2). 



Theorem 8.1.1. The functions cos — — | , sin| -^^^ | ; « > 1 are periodic 

2/ 1 

with fundamental period — and together with the constant function, say — 

n 2 



i7a] . f nm] 
, sin \ ', n : 

I ) [ I ) 
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(the coefficient of Oq in Eq. (8.1.2)), are mutually orthogonal over the interval 
[-/,/], i.e. 



J cos- 



|cos| — : — \dx = { 

I , m = n 



J K ^ J 



sin sin| — : — \dx^ ' 



I j \ I J \l,m = n 

—I 

f . f nm^ ( m7Di\ , 
sin cos lax 

C ( flTDC 1 C ( llTDC 1 

cos MX = sin \dx = 

I [i r I [i ) 



-dx = l 



The proof follows from basic trigonometric identities and is left as an exercise for 
the reader. 

An immediate result of this theorem is that j/the series of Eq. (8.1.2) converges 
uniformly to a function f{x) , which therefore must have a period 2/ (why?), then 
the coefficients are 



1 r ( TlTDC 1 

a„ = - /(x)cos \dx ,n>Q 

-I '^ 

bn =-yf{x)sm\'^\dx,n>l 



(8.1.4) 



To obtain the Eq. (8.1.4) we multiply the equality 



/w=^-Z 



«=1 



nTDc , . nTDc 
a„ cosl + /?„ sin 
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by cos , M > and by sin , « > 1 , integrate over [-/,/J and apply the 

ortliogonality of the trigonometric functions. The formal series of Eq. (8.2.1), 
where fl„,i'„ are taken from Eq. (8.1.4), is called the Fourier series of f{x) , and 

its coefficients are the Fourier coefficients of f(x) . Whether this series converges 
and if so, does it converge to f(x) , is a question of utmost importance that will be 
later investigated. 

Example 8.1.2. Consider the function 

[ X ,0< x<l 

which we extend periodically by f(x + 2l) = f(x) for all x. The result is a 
periodic function with period 2/ , called a triangular wave (Fig. 8.1.1). 




Fig. 8.1.1. A triangular wave. 



Its Fourier coefficients are 



aQ=l;a„ 



21 



2^2 

n n 



[cos(M;r) -l],«>l;ft„=0,«>l 



and the attached formal Fourier series is 



/w~ 



/ 4/ 



n 



Z 



1 



-cos 



-i(in-\Y 



n=l 



(2n - l)7Dc 

I 



As stated, whether the right-hand side converges to f(x) remains to be seen. We 
can however, calculate partial sums S;^ , k>l of the first k terms of the series 
and compare with f(x). A comparison with ^2 and ^3 for I =n , illustrated in 
Fig. 8.1.2, demonstrates how well even ^2 approximates the function. 
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Fig. 8.1.2. The triangular wave for I = 71 \ the approximates ^2 (dashed) and ^3 . 

Before discussing the convergence problem, it should noted that if f{x) is just 

integrable and periodic, valuable information about its Fourier coefficients may be 
already derived. For the sake of simplicity we will assume I = n m Eq. (8.1.2), 
i.e. a function f(x) with period In is considered. Let G denote the set of all the 
linear combinations 



g(x) = — + V [a^ cos(fcc) + B^. &in{kx)\ 



k=\ 



(8.1.5) 



and we search for a function g e G which is closest to f{x) . 
Theorem 8.1.2. If f{x) is an integrable function over \-7t,7i:\ and 



n 

- /^[a„ cos(«x) +/?„ sin(«x)] 



k=i 



where aQ,a^,bi,...,aj^,b^ are given by Eq. (8.1.4). Then /-S'^ < /-g for all 
g(x) of Eq. (8.1.5), i.e. among the linear combinations of the trigonometric 
functions {l,cos(x),sin(x),...,cos(Mx),sin(Mx)} the partial sum of Fourier series is 
always the best approximate to f{x) in the norm defined by Eq. (8.1.1). 
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Proof. 

Using Eq. (8.1.4), we get for arbitrary g{x) e G 



Wf-gf = \\f(x)-^-Y,[AkCos(kx) + B,sin(kx)i dx 

-A *=i J 

= j[/(x)f Jx-2|/(x)U^ + 2]kcos(fcr) + B,sin(fc^)] 

+ [ — + V[A^cos(fcc) + B^sin(Aa:)] 

-A ^ k=\ 

K n 

= ^[f{x)fdx-mQ\-27rY^{a^A,, +^*4) 
+ [ +y]U* cos(kx) + Bi^ sin(Aa:)] 



dx 



dx 



Furthermore, due to cos (kx)dx = sin (kx)dx =n and the mutual orthogonality 
of the functions {l,cosx,sinx,...,cos(Mx),sin(«x)}, we conclude 



2 n 



By adding and subtracting an identical expression to this last equality, one gets 
\\f-gf = \[f(x)fdx-^-7r^(al+bl) 



+ — --m '' • ^' 



k=l 



2 n 



,A^ + '^ + nY,[(a,-A,f+{b,-B,f] 



k=\ 



|[/(x)frfx-^ 



7rY^(al+bl) + 7r 



(«o-A)) 



2 n 



-Y,iia,-A,f+ib,-B,f] 



>o 
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Consequently, the particular choice A^. = a^. ,0<k<n and B^. =b^_ ,\<k<n 
yields 



\f-Sj^ = ^[f{x)fdx- 



^ k=i 



(8.1.6) 



II l|2 II ||2 2 II l|2 

and the proof is concluded since /- g = /-^n +a >\\f-SJ\ , where 



a =7i: 



(«o-A)) 



2 n 



■ + Y,[(ak-A,f+(h-B,f 



k=l 



Since 11/ - SJ\ > , Eq. (8.1.6) implies the Bessel's inequality 



2 n 



1 /■ Z " 



k) 



(8.1.7) 



Example 8.1.3. Let f{x) = x ,-n <x<n . The Fourier coefficients are 



1 " 



. , 2 , 2;r 4cos(n;r) 



and Bessel's inequality yields 



n 

-I' 



4 , 2;r ^ 2,n — 



dx = > + 

5 9 



" 1 



n>\ 



k=l 



or 



-*«»Z^ 



M>1 



*=! 
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PROBLEMS 

1. Find the norms of cos(Mx),sin(Mx),xe^ over [-;r,;rj. 

2. Let /(x) , -oo < X < CO denote an integrable function with period T . Show 
that for arbitrary a,b: 

a+T 



(i) \fix)dx= \f(x)dx 

T 

T a+T 

(ii) \fix)dx= \f{x)dx 

a 

a+T h+T 

(iii) f(x)dx= \ f(x)dx 



3. Let f(x) = x,-7r<x<7r. Extend the function periodically to all x and 
obtain its formal Fourier series 

4. Let f(x) = sin x , < x < ;r . Extend it periodically, obtain its Fourier series 
and compare between /(x) and the partial sums 82,82,8^ over the interval 
0< x<7r . 

5. Find the Fourier series of /(x) = Isinxl ,-n < x<n , extended as a periodic 

function with a period 2n . 

6. Obtain the Fourier series generated by the following functions: 

(a) /(x) = x\-l<x<l ; /(x + 2) = /(x) 

(b) /(x) = cos^x , -;r< x<;r ; /(x + 2;r) = /(x) 

/ N ^/ X cos x,0<x<;r j., ,, ., , 

(c) /(x)= ; f{x + 2K) = f{x) 

[0 , -;r<x<0 

f cosx , < X < ;r 

(d) /W= . ' ; /(x+2;r) = /(x) 

[sm X ,-n <x<\) 

-3 

7. Obtam the Bessel's inequality for /(x) = x,-;r<x<;r. 

8. Show that if the Fourier series of a continuous function /(x) over the interval 
-I < x<l , converges uniformly to /(x) , than the Parseval's identity 



1 r ^ °° 

-\[f{x)fdx = ^ + ^{al+bl) (8.1.8) 



«=i 
holds. 
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8.2 Convergence 

The formal Fourier series generated by a periodic function f(x) may or may not 
converge to the function. In this section we will present a set of sufficient 
conditions, that if satisfied by f{x) , guarantee the convergence of the series. 
First we introduce the concept of piecewise continuity. 

Definition 8.2.1. A function f(x) , defined over an interval [a,b\ except maybe at 

a finite number of points, is called piecewise continuous if for some given 
partition 

a = Xq < Xj < • • • < x,j_[ < x^ =b (8.2. 1) 

of the interval [a,b\, the following requirements are fulfilled for all 
(• : < ( < « - 1 : 

1. f(x) is continuous over the open interval (x,,x,^[). 

2. f(x) converges from inside to finite limits at both endpoints, i.e. 



lim/(x) = /,.^ , lim fix) = /,.;i (8.2.2) 

X-lX: X I X: , 1 



Thus, a piecewise continuous function may not be defined at some partition point 
X, , but it must possess there left and right limits if x, g {a,b) , right limit if x, = a 
and left limit if x, =b . We may also define a piecewise continuous function as a 

function defined and continuous over an interval, except for a finite number of 
points where the function has finite jumps. 

Example 8.2.1. The function 



f(x) 



x,0<x<l 
2,l<x<3 
X, 3<x<4 



which is illustrated in Fig. 8.2.1 is piecewise continuous over the interval [0,4j. 
It is not defined at x = 0,3,4 but has finite limits everywhere at the partition 
points. 
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12 3 4 

Fig. 8.2.1. The piecewise continuous function of Example 8.2.1. 



Example 8.2.2. The function 



fix)- 



1 



2 — x 
4 



,0<x<2 



2<x<: 



illustrated in Fig. 8.2.2 is not piecewise continuous over [0,8j since it has an 
infinite jump at x = 2 . 



Fig. 8.2.2. A non piecewise continuous function. 
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The next result which is presented without proof, provides quite general 
sufficient conditions for the convergence of Fourier series. 

Theorem 8.2.1 (The convergence theorem). Let f(x) and f'(x) be piecewise 
continuous over the interval [-/,/J and let f{x) be extended periodically for 
-oo < X < oo . Then, the generated Fourier series 



-"0 



Zf nTEc] , . f riTVc] 
«„cosl— j + ^«sinl— J 



where the coefficients are given by Eq. (8.1.4), converges to 

-l<x<l 



lim/(f) + lim/(0 

. lix t IX 



(8.2.3) 



uniformly in x . In particular, if / is continuous at x , then 



?r-Z 



a„ cos 



\—rH— 



fix) 



Corollary 8.2.1. Let f{x) and f'{x) be continuous functions over a closed 
interval and let f{x) be extended periodically for all x . Then, the generated 
Fourier series converges uniformly to /(x) . 

Example 8.2.3. Consider the piecewise continuous sawtooth function (Fig. 8.2.3) 
defined over a single period as 



fix)-- 



x + \ , -l<x<0 
X , 0< x<l 




Fig. 8.2.3. The sawtooth function of Example 8.2.3. 
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whose derivative is f'(x) = 1 , x^O . The coefficients of the generated Fourier 

series are Aq = 1 ;a^ = , n > and fo„ = , « > 1 . Therefore, 

riTT 



fix)- --2 



sin(2;rx) sin(4;rx) sin(6;rx) 
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Att 



671 



The average of the right and left limits of /(x) at x = is — and as we see it is 

also the value of the Fourier series. A familiar result is derived by substituting 
1 



/! 



l = l-li-l.l 

4 2 ;rl 3 5 



which implies 

;r , 1 1 

— = 1 — + — ... 

4 3 5 

Similar identities all of which are consequences of Theorem 8.2.1 may be obtained 
by taking other values of x . 



PROBLEMS 

1. Let f(x) denote a continuous periodic function over (-00,00) with period 21. 
What can you tell about the generated Fourier series of f{x) ? 

2. Consider the function 



3. /W = 



X, -l<x<0 
l,0<x<l 

2 — x,l<x<2 



extended periodically for all x and denote its generated Fourier series by 
S{x) . Calculate lim S{x) for a = 0,1,2 . 

3. Generate Fourier series for f{x) = x ,-l<x<l, extended periodically for all 
X , and apply the convergence theorem at x = \ . 
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4. Are the requirements of Theorem 8.2.1 satisfied by f(x) = ^x ,0<x<2?By 
/(x) = Vx ,0.001<x<2? 

8.3 Even and Odd Functions 

Quite often a Fourier series of a function consists either of only cosine or sine 
terms. This occurs when the function is even or odd respectively. Let us recall the 
basic properties of such functions: 

(a) The sum, difference, product and quotient of two even functions are even 
functions. 

(b) The sum and difference of two odd functions are odd functions. 

(c) The product and quotient of two odd functions are even functions. 

(d) Let fix) , g{x) denote even and odd integrable functions respectively over 

the interval [-/,/J. Then 



\f(x)dx = l\f(x)dx , g(x)(jb: = 



-; 



83.1 Even Functions 

Consider an even periodic function f{x) with period 2/ . Since the trigonometric 

I II TT X 1 I Tl TT X 1 

functions cos , sin are even and odd respectively, so are the 

functions /(x)cos , /(x)sin . Consequently, the Fourier 

coefficients of f{x) are 

2 r I TlTT X I 

a„=— /(x)cos \dx , n>0 

^ ^ 

b„=0 , n>\ 

and the expansion 

n-l ^ 

is called a Fourier cosine series. 



(8.3.1) 
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Example 8.3.1. Let f{x) denote the triangular wave of Example 8.1.2. Here 



— xdx = 1 \ a^ = — \ xcos| — ; — \dx = —^ — ^[cos(M;r) — ij , « > 1 





/ 



2 2 ' 



The particular choice / = 1 yields 



/w~ 



j_ A_ 

2 ^ 



cos(;r x) + —— cos(3.^ x) H — - cos(5.^ x) + . 

3^ 5^ 



Since /(x) is continuous, the convergence theorem guarantees the convergence 
of the Fourier series to /(x) for arbitrary x . In particular 



/(0)=0 = 



1 4 



n 



1 + — r + — r + --- 
32 52 



and consequently 



n'^ _y 1 
T"^(2«-l)2 



(8.3.2) 



5.5.2 Orfrf Functions 



Consider now an odd periodic function f{x) with period 2/ . In this case the 

I II TT X 1 ( Tl TT X 1 

functions /(x)cos , /(x)sin are odd and even respectively and the 

Fourier coefficients of f{x) are 



and the expansion 



^„ =-j/(-«)sin -^^ k/x , M>1 



a„=0 , M > 



(8.3.3) 



f(x) ~^K sii 



nn X 
I 



is called a Fourier sine series. 
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Example 8.3.2. Let f(x) = x,-l<x<l extended periodically for all x. 
Integration by parts yields 



1 
b„ =21 xsin{n7rx)d. 


and the generated Fourier sine series is 



'X = ^ ,n>\ 

riTt 



fix) 



n 



sin(;rx) sin(2;rx) + — sin(3;rx) - . . . 

2 3 



Since f{x) is continuous at x = — , the convergence theorem yields 



1-1 

2 n 



1-1 + 1 
3 5 



(8.3.4) 



a relation already obtained in Example 8.2.3. 

Examples 8.3.1 and 8.3.2 illustrate that the function /(x) = x , < x < 1 can be 

expanded as two different Fourier series, namely its Fourier cosine and sine series. 
This does not contradict anything since the cosine series represents the function 
extended as an even function over the complete interval [- 1 , ij , while the sine 
series represents a different function, namely the odd extension of /(x) . In other 

words the two Fourier series represent the same function over one half of its 
domain, but represent a different function over the other half. 

The Fourier cosine and sine series were obtained by choosing specific modes 
for extending a function /(x) , < x < / to an additional interval -/ < x < and 
then carrying the extension periodically for all x . However, sometimes we may 
want to extend a function differently, for example /(x) = , - / < x < . In such a 
case we obtain a mixed Fourier series, consisting of both cosine and sine terms. 



Example 8.3.3. Consider the function 



0,-l<x<0 

X , 0< x< 1 



Integration by parts yield 

1 cos(n;r)-l 



2^2 

n n 



,n>l;b„ 



cos(n7!:) 



nn 



,n>\ 



I.e.. 
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/W~7 + Z 



cos(M;r)-l , , cos(n;r) . , 

■cos(n7rx) sin(n7rx) 



2 2 



nn 



Substituting x = (where the function is continuous) yields the familiar Eq. 

(8.3.2). What do we have at x = - ? 

2 

PROBLEMS 

1 . Which of the following functions are even, odd or neither. 

(a) |x| 

(b) |x + 1| + |x - 1| 

(c) e''' sinx 

(d) e"^'(l + x) 

(e) x^ + x^ 

2. Let f{x) , g(x) denote even and odd smooth functions respectively. Show 
/'(0) = andg(0) = 0. 

3. Show that the derivatives of even and odd functions are odd and even functions 
respectively. 

4. Find the Fourier cosine series for the following functions: 

(a) 



fix)- 



X , 0< x< 1 
2,l<x<2 



period 4 



(b) 



fix)- 



0,0<x<l 

X , 1 < X < 2 , period 6 

l,2<x<3 



5. Find the Fourier sine series for the functions in problem 4. 



9 Elementary Numerical Methods 



Calculus is certainly one of the pillars of modern science and the cornerstone to 
most applications of mathematics in other disciplines. In order to apply its basic 
ideas to real world problems, additional study of subjects such as differential 
equations and numerical analysis is eventually needed. However, the authors feel 
that short introductions to these topics, rather than scare the students, may show 
them the benefits of mastering the mathematical computational tools presented in 
this book. 

9.1 Introduction 

A numerical method is a model for solving a problem for which a solution consists 
of calculating one or several numbers. For example, a numerical method to find an 
approximate to the derivative /'(x) is to calculate f{x + h),f{x-h) for a 
"small" h and take 

, ^ f{x + h)-f{x-h) 
2h 

The quality of this approximation depends on the size of h and we can show that 

f(x + h)-fix-h) , 
hm = / (x) 

h^o 2h 

Usually, we want to approximate a solution to a problem within some desired 
degree of precision or tolerance. The first step is finding a numerical method that 
can perform that task. Then, we design an algorithm, based on this method, which 
is a finite sequence of steps that need to be executed in order to obtain the 
approximate solution. We emphasize the word finite since whether we use a 
pencil, a calculator or a computer, we must confine ourselves to a finite number of 
arithmetic operations. The number of operations may increase if a higher accuracy 
is requested, but it will always stay finite. 

Example 9.1.1. In order to solve the linear equation ax + b = c with arbitrary 
coefficients a,b,c we can use the following algorithm: 
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Step 1. Calculate d =c-b . 

Step 2. If fl = write 'there is no solution' and stop; If a ^ calculate x = — 

a 

and stop. 

Example 9.1.2. Consider the problem of finding the square root of a given a>\ . 

An efficient numerical method for calculating an approximate to vfl , is to create 
the sequence 



_ 1 

XQ-a\x^- — 



a 

Vi+ — 



,n>\ (9.1.1) 



The process of calculating each x^ is called iteration and we often use this term 
for the number x„ as well. Since {x„} is a monotone decreasing sequence (why?) 

and lim x„ = -J a (for details see Section 9.3 below), this method enables us to 

approximate v« to any given degree of precision. The algorithm designed for this 
purpose will be formulated next, after we set an agreeable definition for 
convergence for our problem. 

Given an approximate s to -J a , we measure its accuracy by the distance 
\ s -^a I . Let £■ > denote a prefixed desired tolerance. Unfortunately, we do 
not know v« and are unable to compute L? - v a 1 • However, since 

\ s —a I = 1 .J — va W s + sja \> l-Ja I s — -J a l> 2 I 5 — -J a I 

we may replace the convergence test defined here by the inequality \ s - y/a \< e , 

by the stronger request Is - a\<2e which implies \ s -^ja \< e (why?). The 
algorithm can be represented as follows: 

Step 1. Given an arbitrary positive number a>\ and a desired tolerance £■ > , 
choose an initial approximate to Va (from above), say Xq = a and set 

M = 0. 

Step 2. Use Eq. (9.1.1) to get the iteration x„^.i . 

Step 3. If p:„+i -a\<2e the computation is over and x„^.i is taken as the final 

approximate to Vfl . The total number of iterations needed for 
convergence is n + 1 ; If x^^^j -a\>2e set « ^— « + 1 and go to Step 2. 
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Since in fact lim Xij=^ja , this is indeed an algorithm, i.e. the process must end 

and produce the desired accuracy after a finite number of steps. 

If for example a = \0,e = 10"^ we get x^ - 10 = 20.2500 , x^ - 10 = 3.3889 , 
xj^ -10 = 0.2144 and x^ -10 = 0.0011< 2•10"^ Therefore, the approximate 
X4 =3.1625 satisfies X4-VIO <10" . The number of iterations needed for 

convergence is 4. 

The process used in Example 9.1.2 is a typical iterative process. During such 
process, in order to calculate an unknown quantity a , we usually generate a 
sequence {Xo,Xi,X2,...,x„,...} of iterations that converges to a. Generally, a 
complete convergence requires an infinite number of iterations (as in Example 
9.1.2) which we are unable to perform. Therefore we set some tolerance e>0 
and stop the process when we first obtain |x„ - al < £■ . However, as already stated, 

since a is unknown, this convergence test is impractical. In most cases we 
therefore apply a different test and stop the iteration when the inequality 
U„+i ~xA<£ first occurs. If a relation \x^ —a\< C|x,j^j - x^l can be found (see 

Section 9.2), we simply iterate until k„+i-''^„ < — and thus guarantee 

\x„ —a\<e . 



PROBLEMS 

1. Present an algorithm to obtain the roots of a general quadratic equation 
ax +bx + c = Q . 

2. Find an algorithm to obtain va for arbitrary a > . Assume that the sequence 

of Eq. (9.1.1) converges to vfl . 

3. Show that for arbitrary a > , the sequence created by Eq. (9.1.1) is monotone 
decreasing for all a>\ . What can be derived when a < 1 ? 

4. Archimedes believed at some point that ;r = 3.14159--- is exactly vlO . Use 
Eq. (9.1.1) and problem 3 to show that he was wrong. 

5. Find the number of iterations needed to obtain v9 with accuracy 10" , using 
the algorithm given by Eq. (9.1.1). 



9.2 Iteration 

In this section a detailed case study of an iteration problem and algorithms for 
solving it are introduced. 
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We are concerned with the equation 

x = f{x) (9.2.1) 

where f{x) is a real-valued function defined over the interval {a,b] . Let us first 
present a formal algorithm for solving this equation. By 'solving' we mean, 
finding an approximate value to the exact solution S of Eq. (9.2.1). The algorithm 
presented here is called the Standard Iteration Method (SIM). 

Algorithm 1 (SIM) 

Step 1. Choose a first approximation Xg to the exact unknown solution s , a 
tolerance £ >0 and a maximum number of iterations A^ that we allow 
for the process. Set « = , where n denotes the current number of 
iterations already performed. 

Step 2. Evaluate the next iteration x^^j = /(x,,) . 

Step 3. If |x„^.i-x,J>f and n + l<N set «<—« + ! and go to Step 2; If 
\x„+i - xJ> £ and « + 1 > A^ write 'maximum number of iterations 
exceeded' and stop; If |x„^.i -xJ<£ set n <— « + 1 , write 'convergence 
ended successfully after n iterations with an approximate x^ ' and stop. 

Notice that we have not yet discussed whether Eq. (9.2.1) possesses a solution and 
whether this solution is unique. We also did not state the requirements that need to 
be satisfied to assure that Algorithm 1 provides a real approximate to S . The next 
result provides sufficient conditions for the existence of a solution to Eq. (9.2.1). 

Theorem 9.2.1. 

Let f(x) , a real-valued function over the interval [a,b\, satisfy: 

1. fl < f{x) < b for all x:a<x<b. 

2. f(x) is continuous over [a,b\. 

Then, Eq. (9.2.1) has at least one solution s . 

Imposing the first requirement is essential if Algorithm 1 is to be applied. Indeed, 
if /(x„) g [a,b\ for some n , then x^^^ is simply not defined. 

Proof. 

Let g(x) = X- fix) . Since a < f(a),f{b) <b hy requirement 1, we get 

g{a) = a- f{a)<Q 
g(b)=b-f{b)>0 
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If g(a) = or g(b) = then s = a or s =b respectively is a solution. Otherwise, 

g(a)<0, g(b)>0 and the continuity of g{x) guarantees the existence of 

s :a < s <b such that g(s) = , i.e. s = f(s) . This completes the proof. 

A typical function that satisfies the requirements of Theorem 9.2.1 is illustrated 
in Fig. 9.2.1. Note that there may be more than one solution to Eq. (9.2.1). 




Fig. 9.2.1. Illustrating x = f(x) . 



The next result presents sufficient conditions for the existence of a unique 
solution to Eq. (9.2.1). 

Theorem 9.2.2. 

Let f{x), a real- valued function over \a,b\, satisfy the requirements of Theorem 
9.2. 1 and assume the existence of a constant L : < L < 1 such that 



\f ix^) - f {X2)\< L\x^ - X2\ 



[a,b\ 



(9.2.2) 



Then, Eq. (9.2.1) has a unique solution s in \a,b\. 

If |/(jCi)-/(x2)| < iUi -X2I for all Xi,X2e[a,foJ, we say that a Lipschitz 
condition holds for /(x) . For uniqueness we request a Lipschitz condition with 
0<L<1. 

Proof. 

The existence of at least one solution is guaranteed by Theorem 9.2.1. Let Sy,S2 
denote two solutions to Eq. (9.2.1). Then s^- 82= /(^i) - /(i'2) ■ Consequently 
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and if s^ ^ ij 



^2\ = \f(Sl)-f(S2)\^L'L -■^2 



we divide the inequality by L - ij and get L > 1 which 



contradicts the third requirement of Theorem 9.2.2. Hence, 5; = S2 as stated. 

Let f(x) possess a derivative over [a,b] such that |/'(x)|<L<l. Then, the 

requirement of Theorem 9.2.2 is satisfied. This is easily verified using the mean- 
value theorem, and it is left as an exercise for the student. 

Example 9.2.1. Consider /(x) =0.9x(l-x) , 0< x< 1 . This function is 
continuous and satisfies < /(x) < 1 . Therefore, the equation x = f{x) has a 
solution. To show uniqueness let us check the derivative 
/'(x) = 0.9 - 1.8x , < X < 1 . Clearly |/'(x)| < 0.9 and thus the solution is unique. 

Example 9.2.2. The function f(x) = e^^ is continuous over [0,lj and < e"^ < 1 . 
Therefore, the equation x = e"^ has a solution s : < j' < 1 (Fig. 9.2.1). Since 
/ (0) = 1 the solution may not be unique. However, the functions x and f{x) 
are strictly increasing and decreasing, respectively. Consequently, it is impossible to 
have more than one solution to x = e"^ , i.e. the solution is unique (Fig. 9.2.1). 




Fig. 9.2.2. A unique solution to x = e ^ ,0<x<l. 
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The error at the n -th iteration is defined as the difference between the 
approximate x„ and the exact solution s , i.e. 

e„=x„-s (9.2.3) 

Clearly, the algorithm SIM is meaningful only if x^ converges to s , i.e. if 
lim e^=0 . We will now show that this is the case if f{x) satisfies the 

n— >oo 

requirements of Theorem 9.2.1 and a Lipschitz condition with L<1 . Indeed, 

\^n - ■s| = \f(^H-l)- f(s)\ ^ L\x,,-1 -s\<---< L"|xo - s\ 

Since L<1 we have limL"=0 and consequently lim|x„-.s| = as well. 

Example 9.2.3. Consider the equation x = f(x) where f(x) = l-x /6 over the 
interval [0,lj. The requirements of Theorem 9.2.1 are satisfied and a Lipschitz 
condition with L = 1/2 holds. Since s <l , the choice Xq =0 provides 
|x„ -i'|<l/2" . Thus, to guarantee an error of less then 0.001, ten iterations are 
needed. 

The most important property of an algorithm is whether or not this algorithm leads 
to convergence or simply converges. However, from practical point of view one 
should also consider the speed of convergence. It makes a difference whether 9 or 
9,000 iterations are needed to obtain an error below a given tolerance. For 
example, a popular expression for tt is 

7r = 4\ 1 — + + ... 

3 5 7 

but the student, correctly, will find it unattractive: to reach accuracy of 10" no 
less than a thousand terms are needed. On the other hand, another well known 
relation 

90 "^«4 
that yields 



^-- 



is more appealing: the same accuracy is obtained by adding only six terms! 
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Given an iteration method that approximates the solution s via a sequence 
{x^l^^Q , we now define the convergence rate of the sequence to s ,as follows: 

Definition 9.2.1. Let limx„ =s and let the error sequence {^„}~^o defined by 
Eq. (9.2.3), satisfy 



lim 



K,K>0 (9.2.4) 



for some constants a and K . Then a is called the order of convergence of the 
given method. If « = 1 and A" < 1 the method is called linear. In the case a = 2 
the convergence is called quadratic. The case a < 1 is impossible if the procedure 
converges (why?). 

If the error sequence is such that \£„+i I < ik,, I for sufficiently large n , the order 
of convergence is at least a (i.e. no worse than a ). For example, the standard 
iteration method for solving x = f{x) is at least linear, provided that a Lipschitz 
condition with L < 1 exists for f{x) . However, a careful analysis reveals that if 
f{x) has a continuous derivative, SIM is exactly linear with K = f'(s) . 

Theorem 9.2.3. Let f{x) satisfy the requirements of Theorem 9.2.1 and in 
addition: 

(\) f{x)eC\aM- 

(2) < \f'(x)\ < 1 for all X : fl < X < fo . 

Then, for arbitrary Xg , the sequence {x„},x„ = f(x„_i) converges linearly to the 
unique solution of x = f(x) . In particular 

lim ^^^ = f\s) (9.2.5) 

«^~ £„ 

and since < f'{s) < 1 , the convergence is linear. 

We will now show how by adding very little in terms of computation, we may 
accelerate the convergence of SIM. 

Let f{x) satisfy the requirements of Theorem 9.2.3 and construct the sequence 
{x„},x„ =/(x„_[) . Due toEq. (9.2.5) we have 
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f„^i=(M+^„)f„ , lim^„=0 (9.2.6) 

n— >oo 

where M = f'(s) . If we ignore 0^ for large n , we iiave approximately 

x„+i-5=M(x„-5) 

which implies M = (x^^j ~ ^n+i V^^n+i ~^n^ ^^'^ consequently 



*^«+2 ~ 2x„+i + x„ 



(9.2.7) 



This equation holds only approximately, but the right-hand side can be used as a 
far better approximation to s than x^ . In fact, if we denote 



'^n+2 ~ 2x„^.i + X^ 



■'^« 



it can be shown that 



lim^ = M^ (9.2.8) 



and since 0< M < M <1 , the sequence {x'„ } converges faster than {x„ } to 5 . 
Notice though, that this improved procedure, called the Aitken's method is also 
linear. The improvement is by decreasing the constant K of Eq. (9.2.4) from M 

to M ^ . This by itself already implies that lim s'„ je„ = as can be easily shown 
by the reader. 

Example 9.2.4. Consider the equation x = cos {x)l2 over the interval [0,1] . All 
the requirements of Theorem 9.2.3 are satisfied and let us take an initial 
approximation Xq=1. Table 9.2.1 provides a comparison between SIM and 

Aitken's method and demonstrates the superiority of the latter. While SIM 
requests 15 iterations to obtain six accurate digits, only 6 iterations are needed by 
Aitken's method for the same precision. 
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Table 9.2.1. SIM vs. Aitken's method for x = cos (x)/2 , Xq =1. 



n 


^n 







1.000000 


0.390914 


1 


0.145963 


0.412016 


2 


0.489423 


0.417241 


3 


0.389495 


0.417643 


4 


0.427906 


0.417705 


5 


0.413901 


0.417713 


6 


0.419124 


0.417715 


7 


0.417192 


0.417715 


15 


0.417715 




16 


0.417715 





PROBLEMS 

1. Show that neither requirement of Theorem 9.2.1 is necessary for having a 
solution to X = f{x) . 

2. The following functions are continuous over the interval [0,1] , satisfy 
< /(x) < 1 and a Lipschitz condition exists for each of them. Find the 
solution(s) to X = f(x) and compare the results with the conclusions obtained 
by observing the Lipschitz constants: (a) f{x) = \-x /3 (b) 

f(x) = 0.5 + 0.5sin(10x) (c) f(x) = x (d) f(x) = 0.5 + 0.2tan"'[20(x - 0.5)] 

3. Repeat and solve Example 9.2.4 with initial approximations Xq = 0.75,0.5 . 

4. Explain the advantage of using Aitken's method to solve x = /(x) in the case 



of fix) ■- 



,0<x<l. 



5. Let /(x) = l-x ,0<x<l. Generate the sequence 
Xq = 0.5 and analyze the results. 



= /(x„_i) where 



9.3 The Newton - Raphson IVIethod 



We now present one of the most popular algorithms: a simple, easily manipulated 
and accurate scheme for solving /(x) = . 

Consider the general problem of solving the equation 



/W = 



(9.3.1) 



where /(x) is a real-valued function defined over the closed bounded interval 
I = [a,b] and twice continuously differentiable. Let S denote a solution to 
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Eq. (9.3.1) within the interval and assume that Xq e / is a 'close' point to s , i.e., 
s = Xq + h where h is 'small'. By applying Taylor's theorem we get 

/ 2 



= f(s) = /(Xo +h) = /(xo) + /j/'(Xo) + y /'(c) 



(9.3.2) 



where c is an interim point between Xq and s . Thus, if h is sufficiently small, 

we are justified in ignoring the term (h /2)f''{c) and representing an 
approximation of h as 



A» 



/'(-^o) 



(9.3.3) 



Thus, the value Xj = Xq- /(xq)/ /'(xq) can be expected to provide a better 

approximation to s than Xq . Now we can use Xj as an initial approximation and 

continue the process until we are sufficiently close to the exact solution s . The 
iterative procedure 



^n+l 



" n^n) 



(9.3.4) 



which is based on Eq. (9.3.3) is called Newton - Raphson Method (NRM) and is 
probably the most popular and powerful tool for solving Eq. (9.3.1). 

A geometrical illustration of NRM is given in Fig. 9.3.1: The tangent to the 
curve y = f{x) at (x„,/(x„)) intersects the x- axis at x„+i , which is taken as 
the next approximation to s . 




y=f(x) 



Fig. 9.3.1. The Newton - Raphson method for solving f(x) = . 



244 9 Elementary Numerical Methods 

Example 9.3.1. Let /(x) = 2x'* -x^ -1. One of the solutions of f(x) = is 
s = l. Consider an initial approximation Xq =1.2. By applying Eq. (9.3.4) we 
obtain 



4 3 

OXq JXq 



K^=Xo- '^l\ """ ^ ^ = 1.050673 



and by repeating the process Xj =1.004185 , X3 =1.000031 , x4 = 1.000000 . 

The speed of convergence in Example 9.3.1, as compared to that of SIM or 
even the Aitken's method, is quite impressive. This is no coincidence but follows 
from the following result. 

Theorem 9.3.1. Let /(x) be defined over the closed interval I = [a,b'\, 
satisfying the following requirements: 

1. /(x)e C [fl,^?] , i.e. /(x) possesses at least two continuous derivatives 

within / . 

2. f(s) = for some s:a< s <b . 

3. f\s)^Q 

Then, there exists a neighborhood /„ of s: [s — ri,s + ri], within / , such that 
NRM converges to s for arbitrary initial approximation Xg in /„ . Furthermore, 
the error sequence {f „},£•„= x„ - i' satisfies 

lim%^ = -^^ (9.3.5) 

«^~ el 2 f'(s) 

Theorem 9.3.1. the proof of which is beyond the scope of this book, states that the 
convergence of NRM is at least quadratic. Indeed, if by coincidence the second 
derivative vanishes at the solution point s , the right-hand side of Eq. (9.3.5) is 
zero and the convergence is even faster than the usual quadratic rate as shown in 
the next example. 

Example 9.3.2. Let /(x) = x'* - 2x^ + 1 . One of the solution points of /(x) = 
is s = \ . Let Xq = 1.2 . Then 

Xi=xo-^^^^^ = 0.978704 
4xo - 6x0 



9.3 The Newton - Raphson Method 245 

and X2 =1.000019 , Xg =1.000000 . The second derivative is f"(x) = \2x^ -\2x 
and the convergence rate which is higher than quadratic, is a consequence of 
/'(1) = 0. 

The quadratic convergence of NRM is certainly advantageous. However, there is 
no guarantee that NRM converges for arbitrary Xq , unless Xq e /„ . 

Unfortunately, since we do not a priori know either sox rj , there is no safe choice 
(without further investigation) for Xq , i.e., a random choice of the initial 
approximation may not lead to convergence to s . The next result (presented 
without a proof, which is beyond the scope of this text) removes the necessity of 
getting a good first approximation Xq . 

Theorem 9.3.2. Let f(x), a real-valued function, defined over the closed interval 
I = [a,b] , satisfy the following requirements: 

1. f{x)eC^[a,b]- 

2. f(a)fib)<0. 

3. /'(x)^0,xe/. 

4. f"{x) does not change sign within / . 

5. If C denotes the endpoint (either a or b) with the smaller value of |/'(x)| , 
then 



f(c) 



no 



(9.3.6) 



Then, NRM converges at least quadratically, to the unique solution s of /(x) = 
within [a,i>] , for any first approximation Xg . 

Thus, if the requirements of Theorem 9.3.2 are satisfied, no information about the 
location of s is necessary. Nevertheless, validating requirements 3 and 4 could be 
tiresome, and one of them or both may not hold at all. Therefore, NRM does not 
always provide a full-proof convergence procedure for iteratively solving 
/(x) = . Yet, in many cases, a simple observation may provide an appropriate 

choice of Xn . 



Example 9.3.3. Let /(x) = x -x-l,l<x<2. The first four requirements of 
Theorem 9.3.2 are easily validated. The fifth requirement holds as well: 
/'(1) = 3,/'(2) = 31 i.e. c = land |/(1)//'(1)| = 1/3 < fc-a = 1 . Consequently, 
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the NRM converges to the unique solution of /(x) = at the interval [1,2] for 
any initial approximation Xq within this interval. 

The requirements stated in Theorem 9.3.2 are sufficient conditions for the 
convergence of the NRM. If one or more requirements are not satisfied, the 
process still may or may not lead to convergence. Usually, the iterations will 
converge to the solution s , provided that Ixg - s\ is sufficiently small. Otherwise, 
they may converge to a different solution (i.e. if /(x) = has more than one 
solution), or diverge, as shown in the next example. 

Example 9.3.4. Consider the equation /(x) = sin(x) =0 over the interval 
[-7r/2,7r/2] . It has a unique solution s = which is easily obtained by the NRM, 
if Xq ~ . However, if, for example, we choose as initial approximation the 
positive solution of the equation 2x = tan(x) , i.e. Xq = 1.16556... , the NRM 
yields Xi= -Xq since 

f(xo) _ sin(xo) _ _ 

f (Xq) COS(Xo) 

Similarly, X2 = Xq and the process oscillates. For smaller values of Xg the NRM 
converges to zero and for larger values it converges to a different solution of 
sin(x) = , i.e., a solution outside the given interval. Thus, the NRM does not 

converge for arbitrary Xq e [-7r/2,7r/2] and therefore some of the requirements of 
Theorem 5.3.2. (4 and 5) are not satisfied. 

PROBLEMS 

1. Solve X -15 = using Newton's method. Start with Xq = 2 and stop the 



iteration when x„^, - x,J < 10 



-8 



2. Determine the functions which fulfill the five requirements of Theorem 9.3.2: 

(a) x^ - 6 , - 1 < X < 2 (b) |x| , - 1 < X < 1 (c) x^ ln(x) , 2/3 < x < 2 

3. Solve X -3x =0 using Newton's method with Xg = 0.7 . Show that the 
convergence is slower than expected and provide an explanation. 

4. Use the NRM to solve x-e" "^ =0 with Xq = 1. Find the number of iterations 

I I —8 

needed to get x„ - x„_i < 10 . 

5. Repeat and solve Problem 4 with x^ = 1000 and discuss the two sets of results. 
Is there a significant difference between them? 
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6. Replace /'(x„) by f^^"^'^^^"-^^ in Eq. (9.3.4) to get 



^ _^ fix„)[x„-x„_i] 

/(■^„)-/(-^„-l) 

Use this algorithm (called the secant method) to solve Problem 4 and compare 
the results with those obtained by the NRM. 



9.4 Interpolation Methods 

Quite often it is desirable to replace a given complex function f{x) by a simpler, 
easily manipulated function g(x) , which approximates f{x) over the domain of 
interest. If the approximate g{x) is chosen such that f{x) = g{x) over a given set 
of representative points, it is called interpolator. 

Definition 9.4.1. Consider a function f{x),a<x<b and a set of points 

Xo,X[,...,x,j e [a,i>] . The function g(x) , a< x<b is called an interpolator of 
f(x) at Xo,Xi,...,x„ if 

g(x,.) = /(x,.),0</<« (9.4.1) 

If the number of points is sufficiently large and if they are spread over the whole 
interval, we may expect g{x) to approximate f{x) everywhere, i.e., 

g(x)»/(x),a<x<Z7 (9.4.2) 

A particular useful case is when the interpolator is a polynomial, since 
polynomials are easily manipulated. We next discuss an example of a polynomial 
interpolator, show its existence and derive the accuracy guaranteed by replacing 
an arbitrary function with this interpolator. 

9.4.1 Lagrange Polynomial 

The first question related to polynomial interpolators is whether they exist. The 
most trivial interpolator is a straight line, which intersects a given function at two 
points. This is a linear interpolator shown in Fig. 9.4.1. It does not usually 
provide a useful approximate for the function, unless this particular function is 
almost linear. 
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Fig. 9.4.1. A linear interpolator for arbitrary function. 

The next result confirms the existence of a polynomial interpolator in the 
general case. 

Theorem 9.4.1. Let y = f(x) ,a<x<b. For every data set {(x,-,}',),0 <i<n} 
where Xq,X[,...,x^ e [a,b] are distinct points, there exists a unique « -th order 
polynomial /?„ (x) , which satisfies 



P„iXi) = yi=f(x,),0<i<n 



(9.4.3) 



This is a Lagrange polynomial of order n which interpolates f(x) at 

Xq,X^,...,X^ . 

Proof. 

Consider the n -th order polynomials 

(x-Xo)(x-x,)---(x-x,._i)(x-x,.)(x-x,.+,)---(x-x„) 



/,(X): 



(x,. - Xo )(x,. - Xi ) • • • (x,. - x,._i )(x,. - X,.n ) ■ • • (x,. - x„ ) 



(9.4.4) 



They are well defined since all the points are distinct. Each /, (x) vanishes at all 
the points except at X; , where /, (x,- ) = 1 . Consequently, the polynomial 



Pn(x) = Y,^iix)yi 



(9.4.5) 
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satisfies Eq. (9.4.3). The uniqueness of Lagrange polynomial is easily seen. Let 
q„{x) , another « -th order polynomial, satisfy i7„(x,) = y,- = /(x,) , < / < n . 

Then, the polynomial r„ (x) = /?„ (x) - q^ (x) is of order n and vanishes at m + 1 

distinct points. Therefore r„(x) is identically zero, i.e. p^{x) = q^(x) which 

concludes the proof. 

Naturally, one would expect the quality of a Lagrange polynomial approximation 
to increase with n . However, quite often, this is not the case. It is somewhat 
surprising but easily follows from the next result which is presented here without a 
proof and provides the error obtained by replacing a given function by its 
interpolator. 

Theorem 9.4.2. Consider the function and the data set of Theorem 9.4. 1. Then, its 
Lagrange polynomial satisfies 

/(x)-/7„(x) = — ^-/(x)/("^"(^J,«<x<Z7 (9.4.6) 

(« + l)! 

where /(x) = (x-Xo)(x-Xj)---(x-x,j) and ^^ depends on x and is an interim 
point between a and b . 

Thus, the error obtained by replacing a function by its Lagrange interpolator, 
depends on three quantities. Two of them, (« + l)! and on the {n + l)-th 
derivative of /(x) , are determined by the given function and by the order of the 
polynomial. The third one, /(x) , depends on the structure of the interpolating 
points Xo,Xj,...,x„ . 

Example 9.4.1. Consider the data set 

{(-2.4, 8.3776), (-2.1, 1.3981), (-1,0), (0.8, 1.2096), (2.3, 5.5341)} 



The 4- order Lagrange interpolator, y = x -5x + 4 , which is plotted in Fig. 
9.4.2, does not seem to be ideal for representing the data set, since it oscillates 
while the actual data does not. In fact, the data seems to represent a fast monotonic 
decreasing function, which after reaching a minimum, increases moderately. 
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-2-10123 
Fig. 9.4.2. An unsuccessful Lagrange interpolation for 5 -point data set. 



The possible oscillation of high order polynomials is unlikely to occur when 
low order interpolators are applied. The most popular low order polynomial 
interpolators are cubic splines. 

9.4.2 Cubic Splines 

A cubic spline is a sequence of combined cubic polynomials, connected in a way 
that guarantees continuity and two continuous derivatives of the spline at the 
connecting points: 

Definition 9.4.2. A piecewise cubic polynomial with two continuous derivatives is 
called a cubic spline. 

Example 9.4.2. Consider the function 



/w = 



x"^ + 1 , < X < 1 



2jc^ - 3x^ + 3x , 1 < X < 2 



It is a piecewise cubic polynomial everywhere and the first two left and right 
derivatives at x = l are the same. Hence, /(x) is a cubic spline over the whole 
interval [0,2] . 
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Given a data set {(x,-,/(x,)),0 <i<n} we seelc a function ^(x) which satisfies 
the following requirements: 

(a) s{xi) = /(x;) , < J < M , i.e., s(x) is an interpolator of f(x) . 

(b) s{x) is a cubic polynomial at each interval [x,_i,x,] ,l<i<n. 

(c) s{x)eC^[xQ,xJ. 

(d) /(xo) = A-^„) = 0. 

If such i'(x) exists, it is by definition a cubic spline and approximates f{x) via 
interpolation. Requirement (d) suggests that this interpolator is not likely to 
oscillate. 

Theorem 9.4.3. Given a data set {(x;,}',),0 <i<n} where y,- = /(x,) , < / < « , 
there exists a unique cubic spline (CS) which satisfies requirements (a) - (d). 

Proof. 

Let i(x)be an arbitrary piecewise cubic polynomial over [xo,x„] such that 
i'''(x) is continuous everywhere. Denote M, = s"{xi) ,0<i <n . Since six) is a 
cubic polynomial over each subinterval, ^'(x) must be linear there. In fact we 

easily get 

^,^^^ ^ (x,-x)M,_,+(x-x,_.)M, ^^_^ ^^^^^ ^^^^^ 



By integrating twice both sides of Eq. (9.4.7) and substituting the two boundary 
conditions i'(x,_i) = yj_i , 5(x,) = y^ we obtain 

^ (x,. -x)^M,_i+(x-x,._i)^M,. _^ (x,-x)}',._i+(x-x,._i)y,. 



-Ux, - x,._i )[(x,. - x)M,._i + (x - x,._i )M,. ] , x,._i < X < X,. (9.4.8) 

6 

Clearly, the functions s(x) and ^''(x) are continuous everywhere over [xo,x„]. 
However, it is a priori assumed that ^•'(x) is continuous as well we must also have 
(validation of this claim is left for the reader) 



(x,. - x,._, )M,._i + 2(x,.+i - x,._i )M,. + (x,.+, - X,. )M,.+, 

= , 1</<M-1 (9.4.9) 



+ 6 



Ji - yi-\ y^ - yi 



\ ^i -'■i-i -'■i+i •'■1 y 
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We thus get a system of (n - 1) linear equations with (n + 1) variables 
M 1 ,0 < i < n . By imposing two additional conditions Mq = M„ = (requirement 
(d)) the system can be shown to possess a unique solution given by Eq. (9.4.8). 

Example 9.4.3. Consider the data set {(0,0),(l,l),(2,3)}. The cubic spline that 

interpolates the data and approximates its function is generated by the unique 
solution of the linear system 

M0+4M1+M2 =6 

Mo=0 

Mj =0 

which is Mq = , Mj = 1.5 , M2 = . The CS is composed of two cubic 
polynomials obtained by Eq. (9.4.8) as 

J (x'+3x)/4 

six) = < . , 

(-x^+6x -3x + 2)/4 



PROBLEMS 

1 . Generate the Lagrange polynomial of the data set 

{(-2, 19), (-1,3), (0,1), (1,1), (2, 15)} 

2. Generate the Lagrange polynomial of the data set 

{(0,0),(V4,sin(;r/4)),(;r/2,l),(3V4,sin(3V4)),(;r,0)} 

What is the maximum difference between the polynomial and the function 

sin(x) ? 

3 . Calculate the cubic spline of the data set {(0 , 0) , (1 , 1) , (2 , 1) , (3 , 0)} . 

4. Calculate the cubic spline of the data set {(0 , 0) , (1 , 1) , (2 , 1) , (3 , 1)} . 

5. Repeat and solve Problem 4 where the requirement s"{xq) = s"{XiJ = is 
replaced with s"(xq) = s"(x^) = 1 {s(x) represents the cubic spline generated 
for the data set). Compare between the solutions. 

9.5 Least - Squares Approximations 

Consider some experimental data related to unknown function y = f{x) . If the 
data points resemble linear or quadratic curves, then rather than trying to 
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approximate the data by high order Lagrange interpolator or even by the low order 
piecewise cubic spline, we could find the straight line or parabola that best 
approaches the data. 

9.5.1 Linear Least - Squares Method 

Given a data set {(x,, /(x,)),l <;'<«} and an arbitrary straight line y = ax + b , 
we present the linear discrete least - squares approach as finding the pair 
{a,b) = {aQ,bff) which minimizes the error sum 

n 

E = Y^[f(x,)-ax,-bf (9.5.1) 

/■=i 

The quantity E represents the deviation of the measurements from the straight 
line. In order to get the best linear fit to the data we simply solve 

l^ = 0'|f = (9.5.2) 

da ab 

i.e., a system of two linear equations given by 

n n n 

a2^x^+bY^Xi=^Xiyi 

i-\ i-i i-\ 

n n 

aY^Xl+bn = Y^y^ 
1=1 (=1 

where yj = /(x,- ) , 1 < / < « . 

Example 9.5.1. Consider the following five measurements: 

{(-2,-4.3),(-l,-1.8),(0,0),(l,2),(2,3.8)} 

which may represent for example a small deviation from the straight line y = 2x . 
The linear least - squares (LLS) procedure provides the equations lOa = 20 and 
5b = —Q3, i.e., a = 2 , i> = -0.06 . The approximation's quality is illustrated in 
Fig. 9.5.1. 
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>. 




Fig. 9.5.1. Least - squares approximation for Example 9.5.1. 

9.5.2 Quadratic Least - Squares Method 

Sometimes a quadratic approximation is more likely to represent the data, the 
measurements resemble a curve such as y = ax +bx + c . In this case we 
minimize the error expression 



n 

E = 2_^[/(x,) - axf - bxj - cf 



i=\ 



, , . dE dE dE 
by solvmg — = — = — = ,i.e., 
da db dc 



(9.5.3) 



n n n n 

/■=! /■=! /■=! !=1 

n n n n 

(=1 /■=! /^l /■=! 

n n n 

a^x^+ bY^ X,. +nc = Y yi 



1=1 (=1 
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Example 9.5.2. Consider the 9 - point data set 

{(0,l.l),(0.5,2.2),(l,3.8),(l.5,7),(2,10.7),(2.5,16.3),(3,22.l),(3.5,28.5),(4,38)} 
The equations for determining the quadratic least - squares approximation are 



>-30 




Fig. 9.5.2. Quadratic least - squares approximation. 



PROBLEMS 

1. Find the linear least- squares approximation for the data set 

{(0,1), (2,1), (3, 3), (4,5), (6, 6)} 

2. Use the quadratic least-squares method to approximate the data set of 
Problem 1. 

3. Find the quadratic least-squares approximation for the data set 

{(0,0), (1,1), (3, 3), (6, 6)} 

without solving three linear equations. 

4. (a) Solve Problem 3 for the data set {(0,0), (1,1), (3, 3.2), (6, 6)}. 

(b) Could you anticipate the result of (a) without solving the linear system? 

5. Find the linear least-squares approximation for the function y = e^ between 
x = \ and X = 2 using the function values at x = 1 , 1 .5 , 2 . 
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6. Find the quadratic least-squares approximation for y = e^ between x = 1 and 
x = l> using the function values at x = 1,1.5,2,2.5,2.75,3 . 

9.6 Numerical Integration 

The Riemann integral 

/(/;«,&) =fVw^^ (9.6.1) 

of an integrable function /(x) defined over an interval \a,b\ , cannot always be 
calculated analytically, i.e. it may not be possible to obtain it as finite number of 
standard expressions such as polynomials, trigonometric, logarithmic end 
exponential functions. This is the case even for simple choices of /(x) such as 



e^/x or 4x + x . However, we can always approximate l{f\a,h) using one 

numerical procedure or another. If we apply a given algorithm for evaluating the 
integral, it is then advantageous to be able to bound the error obtained by using the 
specific method. 

9.6.1 The Trapezoidal Rule 

Let an integrable function y = /(x) be defined over the interval [fl,fo] and 
consider the subintervals generated by 

a = Xq < Xj < . . . < x„_[ < x„ = ^ (9.6.2) 

Over each [x,_i,x,] the function is approximated by the secant connecting 
between (x;_i , y^_^ ) and (x, ,y{) , and the area 



f{x)dx 



is approximated by area of the trapezoid defined by 
(x,._i,0),(x,-_i, j,-_i),(x,-,}',.),(x,.,0) , i.e., by 

The whole integral is thus approximated by 
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n 

T„=Y, (^i - ^M )[/(^,-i + fi^i )] (9-6.3) 



The procedure given by Eq. (9.6.3) is called the trapezoidal rule. Its error estimate 
is our next result, given here without a proof. 

Theorem 9.6.1. If /(x)e C [a,b] and if Xq,X],...,x^_[,x^ are evenly spaced, 
then 

/(/;«,^)-r„=-^^^^/''(a (9.6.4) 

where h is the length of each subinterval and ^ an interim point between a 
and b . 

Hence, if I f"{x) \< M for all xe [a,b] then 

\I{f;a,b)-T^\< — h 

Example 9.6.1. Let f{x) = e^ , < x < 2 and we apply the trapezoidal rule to 

evaluate e'^dx using 20 equal subintervals. Here b- a = 2 , h = 0.1 and since 

Jo 

f"(x) = e^ we may take M = e ~ 7.389 . The error bound obtained is 0.0123. In 
fact, the exact integral is e - 1 = 6.389 while Tjq = 6.394 , providing a real error 
of only 0.005. 



9.6.2 Simpson Rule 

The trapezoidal rule basically replaces /(x)by its linear interpolator between 

each two consecutive points, before performing an exact evaluation of the total 
area of the generated trapezoids. The next method goes one step further and 
replaces the function between x,_[ and x,^; by a second order Lagrange 

polynomial which interpolates f{x) at the three consecutive points x,_i,x,,x,^.i. 
The method is called Simpson Rule and in order to apply it we divide the interval 
[a,b] into an even number of subintervals. 

The second order Lagrange interpolator is (Eqs. (9.4.4) and (9.4.5)) 
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(X,_l - X,- )(X;_1 - X,+l ) (X; - X,_i )(X,. - X,+l ) 

+ , (----■;;- ---^ f(x,.,) (9.6.5) 

If x,+j - X; = X; - x,^j = /i we obtain (left as an exercise for the reader) 

r "'fix)dx » r "'pix; x,_i , X,- , x,+i )c/x = -[/(x,_i ) + 4/(x; ) + /(x,+i )] (9.6.6) 

If the partition of [a,b] is composed of equally spaced points 

a = Xq < Xj < . . . < x„_i < x^=b 

where n = 2m, i.e., an even number, the integral, using Simpson rule, is 
approximated by 

Hf;a,b) » 5„ =|[/o +4/i +2/2 +4/3 +2/4 +... + 2/„^2 +4/„_i +/„] (9.6.7) 

with an error 

/(/;«,/.) -5„=-^^^^/(^>(a (9.6.8) 

where ^ is some interim point between a and b . Obviously, Eq. (9.6.8) is 
applicable only if /(x) is sufficiently smooth, which here means possessing four 
continuous derivatives. 

Example 9.6.2. Let /(x) = sin(x) , < x < .tt and consider Simpson rule for 
calculating /(sin(x);0,.^) using 10 equal subintervals. Thus, b-a = tt , h = nllQ 

and since /* '(x) = |sin(x)| < 1 , the error (in absolute value) cannot exceed 



^^^1^ = 0.00017 
180 

However, the exact integral is 2 while the Simpson approximation is found as 
2.0001 1 with an actual error of 0.0001 1 . 
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Since the error bound of Simpson rule contains h at the numerator and 180 at 
the denominator an enormously small error is expected even if h is not too small 
(see previous example where h~0.3). However, if f^ \x) happens to be 
unbounded, the method may not perform well and other algorithms should be 
considered. 

Example 9.6.3. Consider f(x) = ij ^jx , < x < 1 which we define as at x = . 
The function is integrable over the interval [0 , 1] and 



m] 



dx = 2 



Applying Simpson rule with 10 equal subintervals provides Siq =1.60 which 

differs by 20% from the exact value. By decreasing h to 0.05 we obtain 
S20 = 1 .72 which still differs significantly from the correct answer. 

9.6.3 Gaussian Integration 

Since in many applications we replace the integrated function by some 
approximating polynomial, it might be beneficial to apply integration method 
which provides exact value whenever the integrand is a polynomial. This 
motivated the Gaussian integration method which is given next. 

Theorem 9.6.2. Let y = f{x) ,-\< x<\ . Then, for each integer n there exist a 
set of distinct points {x,- , - 1 < x,. < 1 ; 1 < ; < «} and a set of weights 
{wj > , 1 < J < n] such that 



f(x)Jx = £w,./(x,.) (9.6.9) 



/=i 



provided that /(x) is a polynomial of order <2n-\ . Furthermore, the two sets 
are unique. 

The proof of this theorem is beyond the scope of this book. Tables of {x, , w, } 
for various values of n can be found in every textbook of mathematical tables. 
The integration scheme presented by Eq. (9.6.9) is called the Gaussian integration 
method of order n . 

Since every polynomial is a linear combination of powers of x , it follows that 
in order for Eq. (9.6.9) to be valid it must hold for /(x) = l,x,x ,...,x "~ . 
Consequently we obtain a set of 2« equations that x, , w, need to satisfy: 
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/(x) = 1 : 2 = Wj + W2 + . . . + w„ 
fix) = X : = H-jXi + W2X2 +...H'„x„ 
f(x) = x^ : 2/3 = WiXi^ + W2X2 + . . . + w„xl 



2n-2 



/(x) = x^"-^ : 2/(2n - 1) = w^x^"'^ + WjX^""^ + . . . + ^^_;c 

/(x) = x : = ^1X1 +W2X2 +... + w„x„ (9.6.10) 

As stated by Theorem 9.6.2, this system of 2« nonlinear equations with 2m 
variables possesses a unique solution. 

Example 9.6.4. In order to obtain the second order Gaussian method we need to 
solve four equations, namely 

VVi + W2 = 2 
VVlX; + W2X2 = 

WiXj + W2X2 = 2/3 
vviXj + VV2X2 = 

It is easily seen that if any of four variables is zero, so are the others. This 
contradicts the first equation and consequently x, ;^ 0,w, ;^ ; / = 1,2 . Since 

vviXj = — ^2X2 and w^x^ = —W2X2 we get Xj = X2 . The choice Xj = X2 leads to 
contradiction (why?) and the other choice Xj = -Xj when substituted in the 
second equation, leads to Wi - W2 = . Thus w^ = W2 =1 and finally (third 

equation) Xj = -l/v3 , X2 = l/v3 . 

Example 9.6.5. Let /(x) = cos(x) , -1 < x< 1 . Gaussian integration with two 
points provide 

G2(cos(x);-l,l) = cos(-l/V3) + cos(l/V3) » 1.676 
while the exact integral is 

cos(x)(ix = 2sin(l) ~ 1.683 

In this case second order Gaussian scheme is sufficient to obtain an error of 

0.4% . 
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PROBLEMS 

1. Use the trapezoidal rule with 10 equal subintervals to approximate 

xsin(x)dx . 
Jo 

2. Calculate the integral of Problem 1 analytically and compare with the 
approximated value. Verify Eq. (9.6.4). 

3. (a) Use the trapezoidal rule with 10 equal subintervals to approximate 

f(x)dx , where 
Jo 

4. fix) = ' ^ 

[ 0,x = 

(b) Is the error estimate given by Eq. (9.6.4) valid in this case? 

4. Obtain a sufficient condition for Simpson's rule to provide an exact answer. 

5. Use Simpson's rule with 10 equal subintervals to approximate xe'^dx . 

Jo 

6. How many equal subintervals are needed in the previous problem to guarantee 

accuracy of 10" ? 

7. Use the trapezoidal rule and Simpson's rule - each with 5 equal subintervals, 
and the Gaussian integration scheme with two points to approximate 

xe^^ dx . Compare the results with the exact value. 



10 Special Topics 



Certain topics in this chapter are included to demonstrate the beauty, which some 
people, even scientists, fail to find in mathematics. How many people really care 
whether or not the number e is irrational? After all, in performing everyday 
calculations, this number of royalty is cruelly stripped of most of its digits and is 
treated like a common rational number. Yet, no true researcher can stay indifferent 
in front of this amazingly simple yet brilliant proof of Theorem 10. 1 . 1 . 

Other topics, such as Lagrange multipliers and numerical methods for 
differential equations, are meant to remind the reader, that at the end of the day, 
one of the most important requirements from the scientist is to calculate. Some 
topics from this category of practicality, are included in Chapter 10, and are 
characterized by being powerful computational tools and not beyond the scope of 
this book. 



10.1 The Irrationality of e 

The number e = 2.7 1 8. . . which is the basis to the natural logarithm and one of the 
most special numbers in mathematics, is a transcendental number, i.e., it is not a 
root of any polynomial with integer coefficients. The proof, presented by Hermite 
in 1873, is far beyond the scope of this book. In this section we present a simple 
elegant proof of a less pretentious result. 

Theorem 10.1.1 The number e is irrational. 

Proof. 

Assume the opposite. Then, e = — where p,q are integers, and since e is 
between 2 and 3, we must have q>l . Since 



e=lim|l + — =1 + — + — + ■•• + — + ■ 
n 1! 2! A:! 



M. Friedman and A. Kandel: Calculus Light, ISRL 9, pp. 263 J-281.| 
springerlink.com © Springer-Verlag Berlin Heidelberg 201 1 
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we obtain 



Multiplying Eq. (10.1.1) by q\ provides 

k=0 k=q+l 

and consequently 

^11 1 1 
p(q-iy.-q^Y— = + + + ... (10.1.2) 



k=Q 



The left hand side of Eq. (10.1.2) is clearly an integer. On the other hand, the 

1 



infinite series at the right hand side is bounded by 



(q + 1) 



q q' 



which 



(since o > 1 ) converges to r = < 1 . Thus, the assumption that e is 

(q + l)(q-l) 

rational leads to contradiction and the proof is completed. 



PROBLEMS 



1. Show that the numbers ^|e and J is not rational. 



e + 1 
e - 
1 1 



2. Show that the number e, =lH 1 \- ... is irrational 



3! 5! 



3. Show that the number go =lH 1 \- ... is irrational. 

2! 4! 



10.2 Euler's Summation Formula 

Consider a continuously differentiable function f{x) over the interval < x < ©o 

n 

and denote /^ = f(k) for arbitrary nonnegative integer k. The sum /.fk 

k=0 
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n 

approximates the integral f(x)dx and we are interested in evaluating the 



difference 



k=0 

Theorem 10.2.1 If f(x)e c[o,oo) then 



n " 

Y^f,-\f{x)dx (10.2.1) 



Y,fk-\f^^)dx = -{f^+f,) + ^\x-[x]--Y{x)dx (10.2.2) 



k=Q 

where [x] denotes the largest integer less than or equal to x . This is Euler's 
summation formula. 

Proof. 

The relation 

n ^ n n 

Y, \kf'(x)dx = Y,k{ft -A-i) = -XA +(« + !)/« (10-2-3) 

k=\ k-\ k=\ k=Q 

can be easily verified and since A: = l + [x] for every xe[k -l,k) , we may 
rewrite 

„ n 

-J^f,+(n + 1)/„ = j(l + [x])f'(x)dx (10.2.4) 

*:=0 

Also, integration by parts provides 

n n n 

^xf'(x)dx = xf(x)(^-^f(x)dx = nf„ -^f(x)dx (10.2.5) 

c 

By subtracting Eq. (10.2.4) from Eq. (10.2.5) we get 
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Y^f, -^f(x)dx = f„ +^{x-[x]-l)f'(x)dx (10.2.6) 

k=0 

and Eq. (10.2.2) follows by an appropriate rearrangement. 

If in addition to being in C[0,oo) the function f(x) is also positive and 
monotonic increasing, than the left-hand side of Eq. (10.2.2) is positive. 
Furthermore, the series 



n " 

^n ='Yfk -j f(x)dx , n>0 



k=Q 



is monotonic decreasing (why?) and its elements are bounded below by 0. 
Consequently it converges when n ^oo . But so does the series {/„ } . Therefore, 



the integral x - [jc] \f\x)dx converges as n 



— > "SO . 

Example 10.2.1. Let f(x)= ,0<x<oo and replace n by m-1 in 

x + l 

Eq. (10.2.2). Then 

n-l 



1 , ,1 ,„.„^_l , 1 f ?W ^.._1 , 1 fgW 



l + - + --- + --ln(«)=- + ^- { -^^^^dx = - + ^-\^^dx 

J (1 + / 



2 n 2 2m J (1 + x) 2 2n -I X 

where g(x) = x-[x] satisfies g{x) = g{x-\) and |g(x)|<— for all x . As 

stated, the left-hand side converges and we obtain 

limf 1 + - + ••■ + - - ln(M) I = - - [^^dx (10.2.7) 

n^A^ 2 n j 2 J x^ 

The constant at the right-hand side of Eq. (10.2.7) denoted by y , is called Euler's 

constant. Since <g(x)<— we easily get < y < 1 but a more detailed 

computation yields y = 0.57721 

Application 

An important application of Euler' s summation formula is deriving the Stirling 's 
formula 
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lim 



frA" ^Inn _ 



(10.2.8) 



This nontrivial relation is enormously used in many sciences, especially in 
analysis and statistics. Quite often we need to calculate or estimate an expression 

composed of one or several factorials, for example — '- '-^ — . In such cases 

[(3«)!]' 

our only option is usually Stirling's formula. To obtain it, we first introduce 
Wallis ' formula 



n 



lim 

2 n— >~ 



2-4-6---(2«) 



1 



2« + l 



l-3-5---(2«-l)_ 
which will be derived as a consequence of the following lemma. 
Lemma 10.2.1. For every integer « > 2 the inequality 



(10.2.9) 



2-4-6---(2m) l-3-5---(2w-l) ;r 2-4-6---(2«-2) 



l-3-5---(2« + l) 2-4-6---(2m) 2 1- 3-5---(2«-l) 



(10.2.10) 



holds. 

Proof. 

Recall that 



sin"(x)(ic 



2-4-6---(w-l) 
1-3-5---W 

l-3-5---(n-l) n 



odd n 



2-4-6---W 



■ , even n 



(10.2.11) 



Since < sin(x) < 1 for all x:0<x<— we get sin^""^'(x)< sin^" x<sin^" '(x) 

for all ne N , xe (0,.^/2) . Integrating this inequality over the interval [0,.^/2] 

and applying Eq. (10.2.1 1), provides Eq. (10.2.10) which completes the proof. 

Next we verify Eq. (10.2.9). The left inequality in Eq. (10.2.10) implies that 
(how?) 
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2-4-6---(2w) 
l-3-5---(2«-l) 



1 K 

< 



2n + l 2 



while the right one implies 

2n TT 
2n + 1 ' 2" 



2-4-6---(2w) 

1-3-5---(2m-1) 



1 



2m + 1 



By combining these two results we get 



2n TT 



2n + l 2 



2-4-6---(2w) 
1-3-5---(2m-1) 



1 ;r 
< 



2« + l 2 



and Wallis' formula easily follows when « — ^ 00 . 

To get Stirling's formula we apply Euler's summation formula for 
fix) = ln(l + x) . By virtue of Eq. (10.2.2) 



In 1 + In 2 + . . . + In M = In xdx + 



1 



n n 

M = In xdx H — In M + 



gW 



dx 



n + -]\nn-{n-l)+[^^dx (10.2.12) 



where as before g{x) = x-[x] . Dirichlet's test for improper integrals 

guarantees that dx converges (why?). Therefore, the sequence 

J X 



1 



a„ =ln(M!)-| n-\ — |lnM + n 



(10.2.13) 



converges to A = lim a^ = lim 



r n ^ 

1+ l^Jx 

J X 

V 1 



Finding A is simple. Indeed, by 



applying Eq. (10.2.13) twice, for n and for 2m + 1 , we get 
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21n[2-4-6---(2M)] = 21n[2"(l-2-3---n)] = (2n + l)ln(2M)-ln2-2M + 2<3„ 

and subtracting the second equality from the first one, yields 

[2-4-6---(2m)] ,^ ,,, r, 1 ^ , , . . 

In ^ /-' =(2w + l)ln 1 +l-ln2 + 2fl„ -«,„+, 

[l-3-5---(2«- 1)72^571] I 2« + lj 

f 1 V"^' 1 
Since lim 1- = — , the right-hand side of the last equation approaches 

n->~l^ 2n + \) e 

A -In 2 when «^oo, while the left-hand side, by virtue of Wallis' formula, 
approaches Ini-yJ/r/l). Consequently A = ^J27^ and Stirling's formula finally 
follows fromEq. (10.2.13) (how?). 

Example 10.2.2. Let A„ = ^^^ . Then 

(3m)! 

/i„ — 



e 



3n 



■yj^TTn 



V3-3'" 



which yields lim A„ = . 



PROBLEMS 



1. Use Stirling's formula to estimate the binomial coefficient 



'2n. 

for large n . 
n 



2. Find lim — '- 



3. Obtain the final part of Stirling's formula from Eq. (10.2. 13). 



4. Show that hm 



1 In 

\ + + ...+ 



exists for < « < 1 . 



5. Use Wallis formula to approximate 



2" n" \-a^ 

40 • 42- • -78 -80 



39-41---77-79 
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10.3 Lagrange Multipliers 

In order to present the next special topic - Lagrange multipliers, we shall first 
briefly introduce the concept of multi-variable functions, continuity of these 
functions and partial derivatives. 

10.3.1 Introduction: Multi-variable Functions 

The set 

9t" ={(xi,X2,...,xJ,-oo<x,. <oo,1</<m} (10.3.1) 

is called the real n - dimensional space. It consists of all the elements 
(jCpjCj,...,^^)- called points, where each x,- is a real number. Two points 
X = (x[,X2,...,x„) and y = [yi,y2,...,y„) are equal if and only if they are 
identical, i.e., if x, = y, ,1 < ; < n . We define addition of two points and 
multiplication of a point by a real number A as 

x+ y = (xi + yi,...,Xij + y^) ; /k = (/lx[,...,/ix„) , -oo</l<oo (10.3.2) 

Subtraction is defined by x- y = x + (-l)- y and the distance between two 
arbitrary points x, y as 



l^ixi-yif (10.3.3) 



V '=1 

which equals if and only if x and y are identical. Let D denote a subset of 

3i" . A rule / which assigns a unique real number z to each point of D is 
called a function. This n-variable (or n-dimensional) function is a relation from 
D into 9t . For the sake of simplicity we shall assume that D is an open set, i.e., 
for each XqE D there exists S(xq) > such that \x - Xq\ < S ^> x e D. In other 
words, Each point in D has some sphere around it which is completely in D . 

o 1 

Example 10.3.1. The relation z = x is a two dimensional function defined 

y 

for all real x,y such that y ^0 . 



Example 10.3.2. The relation u = yx + y + z defined over 9t is not a 
function since each point [x,y,z) provides two values for u . 
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Definition 10.3.1 (continuity). A function m = /(X[,x2,...,x„) defined over 

D c 9i" is continuous at a given point Xg = (xio,...,x„o)e D if for an arbitrary 
£• > , we can find ^ > such that 

||x - XqI < J ^ ||/(x) - /(xq|| < e (10.3.4) 

In this case we also write lim f{x) = /(xq) . 

Example 10.3.3. The function f(x,y) = x +sin(xy) is defined and continuous 
over the whole x- y plane. 

Example 10.3.4. The function 

' 1 

x 
f(x,y) 



2 ^Ax,y)^ (0,0) 

X + y 



l,x=y = 



is defined everywhere in 91 and continuous everywhere except at (0,0) . 
We now introduce the concept of partial derivative. 

Definition 10.3.2. Let /(xi,X2,...,x„) be an «-dimensional function defined over 
£)c9t"andlet Xg = (xio,...,x„o)e D.lf 

,. /(-*^10'---'-*i-l,0'^i +^'^i+l,0'---'-*no)~/(-*10'---'^i-l,0'-*^i'^i+l,0'---'-*no) (]()'i ^^ 

h->a h 

exists, we say that /(xj,X2,...,x„) has a first partial derivative (or simply a 
partial derivative) with respect to Xj at the point Xg =(xio,...,x„0) . The limit is 

denoted by -— (x,o,...,x„o) or by / (xio,...,x„o) . 
aXj 

Thus, a partial derivative with respect to a specific variable x, is obtained 
by treating the other variables as constants. In other words, during the process 
of calculating the partial derivative, / is considered function of the single 
variable x, . 
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2 3f 

Example 10.3.5. Let f(x,y,z) = x y-cos(xz). The partial derivative — is 

dx 
obtained by taking constant values for y, z , i.e., 

-- = 2xy + z sin(xz) 
dx 

If X, z are kept constants, we obtain the partial derivative with respect to y as 

dy 
The third partial derivative with respect to z is 

— = sin(xz) + zx cos(xz) 
dz 

Second order partial derivatives are obtained using the same principle of treating 
all the variables but one as constants. Given an arbitrary /(jCj,X2,...,x„) , we 
define its second order partial derivatives as 



3V _ d 
dxfixj dxi 






(10.3.6) 



We often write the left-hand side of Eq. (10.3.6) as /^^ . The relation between 
/j. J. and /j J. is given next without proof. 

Theorem 10.3.1. If /jj and /^ ^ are continuous then 

/x.., =/.,.„ (10.3.7) 



X 

Example 10.3.6. Let fix,y) = xe^' . Then 

y 



y y 



and clearly 
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f =f =ey+ — 

J xy J yx 2 

y 

Definition 10.3.3. A function /(X[,X2,...,x^) is said to possess a local maximum 
at Xq =(xiq,...,x^q) if there exists an £>0 such that 

\\x-Xo\\<e^f(x)<f(xo) (10.3.8) 

and a local minimum if an e >0 exists such that 

\\x-Xo\\<e^f(x)>f(xQ) (10.3.9) 

A basic property of local maximum (minimum) points is given next. 

Theorem 10.3.2. A necessary condition for /(xi,X2,...,x„) to possess a local 
extremum (maximum or minimum) at Xq = (X[q,...,x^q) is 



dX: 



(xo) = , l<i<n 



(10.3.10) 



PROBLEMS 

1 . Find the domain of definition and the points of discontinuity for: 

(a)/(x,^) = ^^ 



(b) f{x,y)- 



X -y 
J_ 

xy 

1 , 



xyitO 



xy = 



2. Calculate the first partial derivatives and determine their domains of definition 
for: 



(a)/(x,^) = - + V?+^ 



(b) f(x,y) = ^^^ + ^x^ + y^-z^ 

X 

3. Calculate all the second partial derivatives for: 

(a) f{x, y) = sin(x3') + cos(x + y) 

(b) f(x,y) = tan(xyz) 
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10.3.2 Lagrange Multipliers 

The problem of maximum or minimum search for a given function is carried quite 
often under constraints. For example, if r(x, y, z) denotes the temperature at an 

arbitrary point {x, y, z) , we may be interested at the minimum (or maximum) of 

T(x,y,z) over the surface of some three-dimensional body given by 

f{x,y,z) = . The minimization process is thus restricted to a subset of the 

definition domain of T(x, y, z) ■ The Lagrange multipliers method is a simple 

elegant procedure for solving extremum problems under constraints. 

Definition 10.3.4. Let f(x),gi(x),g2(x),...,g^^j{x) denote continuous functions 
defined over a domain D in 3i" . If for some x e D that satisfies 

gi(x^) = ,\<i<m 

there exists a neighborhood Dq <^ D such that f(x)>f(x ) for all xe Dq for 
which 

g,.(x)=0 , l<(<m (10.3.11) 

the point x is called a relative minimum of f{x) under the constraints given by 

Eq. (10.3.11). We similarly define a relative maximum and in either case refer to a 
relative extremum. 

Example 10.3.7. Let f{x,y) = x +y . We search for a relative minimum 
of f{x,y) under the constraint g{x,y) = x- y-\ = Q . This can be done by 
finding the minimum of F{x) = f{x,x-\) = x +{x-\) =2x -2x + \. We 
differentiate F{x) twice and obtain the minimum at x = 1/2 . Consequently 
f(x,y) attains a relative minimum at (1/2,-1/2). 

Notice that we first eliminated y from g(x,y) = Q , substituted it in f(x,y) 
and then performed a standard procedure to find the extremum points. Usually, 
this approach can be tiresome and impractical. Instead, we present the Lagrange 
multipliers method implemented on the following example. 

Example 10.3.8. Consider all the boxes centered at (0,0,0) with edges each 
parallel to one of the axes, which are contained inside the ellipsoid 

2 2 2 

g{x,y,z) = ^ + ^ + ^-l = Q (10.3.12) 

a b c 
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If (x,y,z) denotes the corner with three positive coordinates, the box volume is 
V{x, y, z) = 8xyz- Since the corners of the box with the largest volume Vg must be 
located on the ellipsoid itself (why?), it is evident that to obtain Vg we have to 
maximize V under the constraint given by Eq. (10.3.12). We will now present a 
technique called Lagrange multipliers for treating this problem. Define 

H(x,y,z,Jl)=Vix,y,z) + Agix,y,z) (10.3.13) 

where A is an additional variable, and perform an unrestricted extremum search 
for H{x,y,z,X) , i.e. solve 

^ = 0,^ = 0,^ = 0,^ = (10.3.14) 

dx dy dz dA 



or 



dV ,dg , 2Ax ^ 
ax ax a 

dV ,dg , 2Ay „ 
T— + A^ = 4xz + —f- = 
ay ay b 



dV ,dg_, 2Az_^ ^1«-™ 



-, - + A^ = 4xy + ^ = 
az az c 



111 
a^ b^ c^ 



due 
Simple algebra yields the unique solution of x = —j= , y = —j= , z = —j= ■ 

V3 V3 V3 

Obviously, this solution satisfies the constraint itself, which is the last part of Eq. 
(10.3.15). Is it also a relative maximum? The next result provides the answer. 

Theorem 10.3.3. Let /(x) , g(x) denote continuously differentiable functions 



over a domain D in 9t" and assume Vg(x) = 



ydx^ dx„j 



i^Q ,xe D , i.e., at 



each arbitrary point xe D at least one partial derivative of g{x) is nonzero.. If 
f{x) has a relative extremum under tl 
then there exists a number A. such that 



f(x) has a relative extremum under the constraint g(x) = at a point x e D, 
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m^^;^M^ = , i<i<n (10.3.16) 

Since the equation 

8(x°) = (10.3.17) 

holds by definition, we end up with a system of « + 1 equations and m + 1 
unknowns, namely Xj , ^2 , . . . , x„ and A. . 

Thus, if a problem possesses a relative extremum and if the system composed 
of Eqs. (10.3.16-10.3.17) has a unique solution, this solution must be a relative 
extremum. This is exactly the case in Example 10.3.8. 

Proof. Since Vg(x ) ^ we have — ^^t at least for one k between 1 and 

ds(x^) 
n . For the sake of simplicity assume — ^ ^ . Therefore, by implying the 

dx„ 

implicit function theorem, we can uniquely solve g(x) = at some neighborhood 
of X and get 

x„ =/j(xj,X2,...,x„_i) (10.3.18) 

where h is continuously differentiable. Denote x =(xi ,...,x„_i,x„) = (x*,x„) 
where x* =(xj ,...,x„_i) . Then the function /[xi,...,x„_i,/!(xi,...,x„_i)] is 
continuously differentiable at a neighborhood of x* in SR"^ and attains an 
unrestricted extremum at x* . Consequently, by using the chain rule 

a/(x°) , a/(x°) dh(x*) 



dXj 3x„ dxj 



--0,l<i<n-l (10.3.19) 



Now, the identity g[xi,...,x„^j,/j(xj,...,x„_j)] = implies 



dXi dx„ dx: 



0,l<i<n-l (10.3.20) 



09(x ) 

which, since — ^ ^ , yields 
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dxi ax,. / ax„ 

By substituting in Eq. (10.3.19) we obtain that Eq. (10.3.16) is valid with the 
constant 



This completes the proof of Theorem 10.3.3. 

Application 

The next example of maximizing the area of a triangle with constant given 
circumference, though simple, demonstrates the potential existing in the Lagrange 
multipliers method. 

Example 10.3.9. Consider all the triangles with sides a,b,c such that 
a + b + c = 2p for some given p . The area of each triangle is given by Heron's 
formula as 



S=^p(p-a)(p-b)(p-c) (10.3.23) 

and the problem of finding a triangle with maximum area is therefore maximizing 
S under the constraint a + b + c-2p = . Clearly, we may maximize S instead 
of S . Therefore, using the Lagrange multipliers method, we search for an 
unrestricted extremum point for H = S + Ma + b + c- 2p) and get the four- 
equation system 

dH 

—- = -p(p - b)(p -c) + A = 
da 



— — = -p(p - c){p - a) + /I = 
ab 



~\TT 

= -p(p - a){p -b) + X = Q 
ac 

= a + b + c — In = 



(10.3.24) 
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2p 
This system possesses the unique solution, a =b = c = — , i.e. an equilateral 

triangle and it is a relative maximum solution. The value of the single Lagrange 

3 
P 

multiplier, A, = - — , is of no meaning or use to us. 

We will now present the Lagrange multipliers procedure in the case of several 
constraints. In order to find a relative extremum of f{x) under the constraints 

g,(x) =0 , 1 < ( < m , we search for unrestricted extremum of the (n + m)- 
variable function 



H(x;Ai,A^,...,Aj = fix) + Y,^igi(x) (10.3.25) 

where the Lagrange multipliers A, ,l<i<m are additional variables to the 
original Xj , Xj , . . . , x„ . The system that needs to be solved consists of the following 
(n + m) equations: 

^ = l^ + £^.¥^ = . l^'^« aO-3-26) 



dx: ■"* dx: 



and 



^ = g{x) = Q , \<j<m (10.3.27) 

Notice that due to Eq. (10.3.27), any solution of this system must satisfy all the 
constraints. The question of existence and nature of the solutions to Eqs. (10.3.26- 
10.3.27), is answered by the following result, which is an extension to Theorem 
10.3.3, given here without a proof. 

Theorem 10.3.4. Let f{x),g^{x),g2{x),...,g^{x)A&i\o\s, continuously 
differentiable functions over a domain D in 9t" and assume that for every xe D 
there exist distinct numbers i^,i2,...,im between 1 and n such that 

5(gpg2^---^gm) (^)_^Q (10.3.28) 

9(x,-,x,.^,...,x,._) 

where the left-hand side of Eq. (10.3.28) is the determinant 
Ittji^ \,l< j,k<m ,a 11^ = — -. Then, if f{x) attains a relative extremum at a 
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point X e D , under the constraints g, (x) = , \<i<m, there exist numbers 
A ,l<i<m such that 



*<£!),y,^*i<f!l.„ , ,,,<_„ „o.3.29) 

dX: ^ ■' dX: 

I j=l I 

This theorem illustrates that the general problem of finding a relative minimum or 
maximum of f{x) can be usually replaced by a more familiar problem of finding 

unrestricted minimum or maximum respectively. 

The next example is a typical problem in Linear Programming, where some of the 
constraints are formulated as inequalities rather than equalities. We first define 
equivalent constraints given as equalities, then use the Lagrange multipliers procedure. 

Example 10.3.10. Consider a problem of minimizing the function 

under four constraints given by 

Xi + 3x2 + 4x3 - 10 = , X, > , xj > , X3 > 
Three of the constraints are inequalities and can be replaced equivalently by 

X] - Ji = , Xj - ^2 = , X3 - }'3 =0 

The function for which we search an unrestricted minimum is 

//(X[,X2,X3,}'[,}'2,)'3,/lj,/l2,/l3,>i4) = .'^l + 2x2 '^ ^3 

+ Zi(x^ - y^ ) + A2 (X2 - y2) + ■^(Xt, - ^3 ) + -^4 (-'^i + 3X2 + 4x3 - 10) 

and the system to be solved is 

-^ = l + /ll+A4 =0,-^ = 2 + /l2+3/l4 =0, -^ = 1 + ^+4/14 =0 
OX[ 0x2 0X3 

-— = -2Aiyi = , -— = -2Jl,y2 = , -— = -2Aiyj, = 
dyi dy2 dy^ 

—- = xi-yi =0,—- = X2-y2=0,—- = x^-y^=0 
oAi 0/i2 oAj 

= X, + 3x2 + 4x3 - 10 = 
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Since H can be rewritten as 

then, in order to get a minimum we must have A, < , 1 < / < 3 . The first three of 
the ten equations yield 



\ = 


-1 


-^4 


^ = 


-2- 


-3/I4 


^ = 


-1- 


-4/I4 



while the next three determine /I,}',- = , 1 < ; < 3 , i.e. /l,x, = , 1 < ; < 3 . If all x, 
vanish then the constraint Xj + 3x2 + 4x3 ~ 10 = is invalid. Therefore, at least 
one of them, say x , is nonzero. We therefore get /I , = . Simple algebra yields 
that in order to guarantee A, < , 1 < / < 3 , we must have A^ =0 . Consequently, 

1 3 5 

Aa = — , At = — , Aj = — 
4 4 4 

and X[ = X2 = . Finally, the constraint Xj + 3x2 + 4x3 -10 = yields Xj = 2.5 
and the relative minimum of Xj + 2x2 + ^3 i^ attained at (0,0,2.5) and equals 2.5. 

PROBLEMS 

1. Find extremum values of X1+X2, where (xi,X2) travels on the unit circle 

2,2, 
X[ + X2 = 1 . 

2. Find extremum values of Xj +X2+X3 where (xj,X2,X3) travels on the unit 

2 2 2 

sphere Xj + X2 + X3 = 1 . 

3. Let a,b denote two positive numbers. 

(a) Find the extremum of /(x,};) = — h— given x +y =1. 

a b 

22 ^ y 

(b) Find the extremum of g(x,}') = x +y given — I — = 1. 

a b 



■2 -.X. 



4. Find the shortest distance from the point (2,0) to the parabola y^ 

5. Let a,b,c denote positive numbers. Find the maximum of x'^y z^ under the 
constraint x-V y + z = \. 
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2 2 

6. Find the extremum distances from the circle x + y =1 to the straight line 
X + 2^-6 = 0. 

n n 

7. Find the minimum of / x, under the constraint y fl,x, = 1 . 

8. Find the minimum of x+ y + z subject to the constraints x + 2y + 3z = 6 and 
x>Q,y>Q,z>0. 
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Chapter 2 
Section 2.1 

1 . The collections (a), (c) are equal setes. The collection (b) is not a set since the 
element -3 appears twice. 

2. li xe A-B then xe A , xi B , i.e., xi B- A. 4. (b) There are 8 elements 
in S . 

7. Let xe A-{BuC) . Then, xe A,xi B<uC . Consequently xi B ,xi C , 
i.e., xeA-B and finally xe (A-B)- C . Thus, A-(Bu C) c (A-B)- C 
and Similarly A - (B u C) 2 (A - B) - C , which completes the proof. 

Section 2.2 

8. (a) By definition (« + 1) =(n + l)(n + 1) which implies 

(m + 1)2 = M • (m + 1) + 1 • (« + 1) = («H « • 1) + (1 • M + 1 • 1) 

and finally leads to (n + 1) = n +2n + l 

9. The inequality n > n + 3 certainly holds for n = 3 since 3 =9>3 + 3 = 6. 
Assume that it holds for some arbitrary n . 

Section 2.3 

3. Define 2' = 2 and 2""" = 2" • 2 , « > 1 . The inequality 2" > « holds for 
« = 1 since 2 > 1 . If it holds for arbitrary n , i.e., if 2" > n , then 

2«+i = 2" • 2 = 2" + 2" > « + M > « + 1 
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r + s 
1. t = 

2 

9. (a) « = 1 : (1 + r)" = 1 + r = 1 + Mr (b) Assume (1 + r)" > 1 + nr for arbitrary 

n , Then, since 1 + r > , we get (1 + r)""^' = (1 + r)" (1 + r) > (1 + nr)(l + r) , 
i.e., 

(1 + r)""^' > 1 + «r + r + r^ = 1 + (« + l)r + r^ > 1 + (« + l)r 

Section 2.4 

m 
1 . Let ^ be a prime number and consider a rational number x = — such that 

n 

m, n have no common factor and m =kn . Consequently, k must divide 

both m and n (why?) which contradicts the given input. 

16. Substitute a = b = \ in Newton's binomial theorem (Eq. (2.4.3)). 

Section 2.5 

6. We consider four cases: (a) x + 5>0,x-7>0,x + 5<x + 7=> no solution. 
(b)x + 5>0,x-7<0,x + 5<7-x ^-5<x<l. The other two cases are left 

for the reader. 

7. The given inequality is equivalent, for all a > , to a + 1 > 2a which holds 
since a^ +\-2a = {a-\f >Q . 

.^ ^, , «/r^; 1 + 2 + .. . + « n{n + \) M + 1 „ , ( n + \ 

10. Clearly, Vl-2-...n < = — - = .Hence, n\<\ 

n 2n 2 12 



Chapter 3 
Section 3.1 

4. The two straight lines x = 2 , x = 3 . 

5. The particular part of the graph is the set 

{(2,2), (3,3), (3,7), (5,5), (5,7), (7,3), (7,5)}. 

6. S =^X,y) — oo<x<oo^j;>0|. 

Section 3.2 

1 . (a) The relation is a function with domain D = {xl x > l| . 

(d) The relation has a domain D = jxl < x < l| and it is not a function. 
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3. The domain is D =\x\x^2,3\ withiange R = \y\-oo < y <-4 ^ y >0^ 

6. (b) X = — (d) Show that the function does not have an inverse by 

4-3y 

obtaining a solution other than Xj = Xj to Xj H = X2 -i . 

I I / 2 , o 

10. (a) strictly increasing 12. (a) f ° g = — ~ — . 

1 + x 

Section 3.3 

2. y = six + X implies [y- x) = x and y - 2xy + x - x = , i.e., y is an 
algebraic function 

7. (a) X = •\/2 + v3 implies x = 5 + 2v6 and \x -5) - 24 = , i.e. x is an 
Algebraic number. 

12. The function is not defined for x<0, i.e., the domain is the single number 

x = 0. 
15. The function /(x)g(x) is odd. 



Section 3.4 

23 
2. For example choose Mq 



7•10"^ 



+ 1 . This is no guarantee that that the 



sequence converges to — or to any limit. 

4. The limit exists and equals . 

l-q 

7. Clearly - + — >— + — = — , - + — + — + ->- + - + - + - = — etc. Thus, 
34442 567888882 

k 
a„ exceeds — for all positive integer k as n ^oo , 

10. a„=l+ 1 + +•••+ =2 which implies 

V 2j U 3) U-1 n) n 

lim a„ =2 . 



Section 3.5 

2. lim(fl„&„c„)= lim ((a„Z7„ )c„ ) = lim (a„/7„ ) lim c„ ={AB)C = ABC . 
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7. Without loss of generality we assume a > . Let m be an integer sucii that 
OT = [flj + 1 which implies m> a . For an arbitrary n> m we get 

n m-1 m-1 / \n-m+l 

a a a a a a a ] 

■ < 7 r- — — > a.v « — > oo 



n\ \m — l)\mm + l n (m — Ij! 

11. A= 12. Not necessarily: {fl„} = {0, 1,0,1,...} and 

p„} = {l, 0,1,0,. . .{are divergent while their sum clearly converges to 1 . 

13. For arbitrary r the sequence a„ = r^ , « > 1 is a sequence of irrational 

n 

numbers which converges to r . 

14. No - use the example of problem 12. 



Section 3.6 

1. 3. (c)sup(53) = -,inf(53) = -l, iim(53) = lim(53) = l. 

4. (d)sup(54) = -,inf(54) = --, iim(54 ) = lim(54 ) = . 

9. The even and the odd terms separately, converge to . This is no guarantee for 
convergence. For example consider the sequence <1, 0,1,— ,0,2, — ,0,3, 

10. The sequence does not converge. sup (5') = 1.1, inf(5) = 10" , 
iim(5) = lim(5) = l. 



Section 3.7 

1. Clearly Aj =l + 'vl = 2 > 1 = Oj . Using Assume a^^j > a,, . We then get 

'^n+i = 1 + V'^n+i > 1 + V^ = '^n+\ (obviously a„ > , « > 1 - why?). 

3. All the sequence elements are obviously positive and a^^j = fl„ + Ja„ > a„ . If 

the sequence converges to some positive A then A = A + -J A which leads to 

A = and to contradiction. 

„ n n n n n 

9. a„ = ... > — = M — ^ oo 

12 n — \n 1 
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Chapter 4 
Section 4.1 

2. For an arbitrary Xg > write 



Vx-/^: 



■Xq 



■Jx + ^jx^ 



from which we easily get that f{x) has a limit at Xq . The case Xq = is 
straightforward. The function does not have a limit at x = -1 since it is not 
defined at the interval between -1 and . 
4. The function has the limit at x = . 

Section 4.2 

1. (b) The function is clearly continuous for x>0 where /(x) = x and for 
X < where /(x) = -x . It is also continuous at x = : Indeed, given £ > 
we look for ^>0 such that |x-0|<^ implies |/(x) -/(0)| < f and the 
obvious choice is S = e . 

3. For any given Xg and arbitrary £ >0 choose d = — . 

Section 4.3 

1. Clearly, 

/"(x)-/"(xo) = [/"-'(x) + /"-2(x)/(xo) + ... + /"-'(xo)][/(x)-/(xo)]. 
Since /(x) is continuous, it is bounded by some M > which implies 

|/"(x)-/"(xo)|<«M"-'|/(x)-/(xo)| 

where nM"^ is constant. Consequently, x— ^Xq implies /"(x) — > /"(xq) as 
well. 

3. The function g(x) = x +sin (x) is a sum of two functions which are 
continuous everywhere. Thus, it is continuous everywhere and so is /(x) as a 
composite function. 

4. (b) X ;^ 1 (c) -oo < X < oo 
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Section 4.4 

, l + Jiy-l r -I 

2. The inverse function is / (y) = and it is continuous over [2,7J. 

3. The inverse is x = '^y - 1 defined for 1 < 3; < oo . 

TT n 

5. The function y = cos(x) , <x< — does not have an inverse since it is not 

2 2 

one to one. For example cos(;r/4) = cos(- njK) . 



Section 4.5 

1. The function is both continuous and uniformly continuous. 

2. (a) The function is continuous, (b) It is not uniformly continuous. 
(c) The function in problem 1 has a limit as x — > 00 . 

3. A function defined over a closed interval and satisfies a Lipschitz condition, is 
continuous over this interval and is therefore uniformly continuous there. 

7. (a) The function satisfies a Lipschitz condition: 

X — }; =x— }'x + xy + y \<2l\x — y\ 



Chapter 5 
Section 5.1 

1 . The derivatives are 3x + 2x , 8 . 

5. The function is not continuous at any point and therefore derivative nowhere. 
7. When two derivatives are identical the corresponding functions differ by a 
constant. 

1 



10. The function has derivative for x > and /'(x) - ^ . 

12. At X = the function has a right derivative and a left derivative 1 . Thus it 
does not possess a derivative there. 

Section 5.2 

1. (c) /'(x)=— 3x" y] X -\ (=-^^ =— 2.5x" yj X 

2vx 
2. (a) fix) = 3(x2 + x)2 (2x + 1) 
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4. The function is one - to - one and consequently the inverse exists and 

l + 3x 

/ 3 2V 2 -2±Vl24 

7. Solve \x +x ] =3x +2x = 10. The solutions are and the 

^ ' 6 

positive one is within the interval [1,2] . 

9. ^ = ^. 
15 

Section 5.3 

1. [sin (x) + cos (x)\ = 2sin(x)cos(x) + 2cos(x)[-sin(x)J = . 
3. The left-hand side of the relation is always positive! (why?). 

. 2/ N • , 2/ N e +e +2 e +e -2 , 

8. (a) cosh (jc) - sinh (x) = = 1. 

4 4 

, ,, , cosh (x)-sinh (x) 1 , • •, , 

9. tanh (x) = r = r and similarly 

cosh (x) cosh (x) 

coth'(x) : 



2 

sinh (x) 



11. (a) The domain is all x for which cosh(x) > , i.e., -oo < x < oo . 

Section 5.4 

1. (a) y" = -sin(e^)e ^ + e" cosCe"^) . 2. The stationary points are x = , x = « . 

4. The function is defined for all XitQ and has a minimum at x = I . 

V 4 

8. Clearly e^ > 1 for arbitrary x > . By the first mean-value theorem 
e^—l = e^—e =xe'' ,0<^<x 
and thus e^ = 1 + xe^ > 1 + x , x > . 

Section 5.5 

1. (a) - 2. (b) -2 4. (b)0. 
4 
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Chapter 6 
Section 6.1 

3 



1. 



a) f(x)dx = 5 (b) The integral still exists and its value unchanged. 





4. No. 



Section 6.2 

1 . (a) Differentiability (or continuity) (b) Continuity. 

Section 6.3 

2. Since f(x) is continuous and g{x) monotonic increasing and bounded by 
0.001 , all functions are integrable (Theorems 6.3.1, 6.3.5, 6.3.6). 

5. Use Theorem 6.3.6. 

Section 6.4 

r ^ N 7.^ ^, ^ (a + 2) -3 , , . 1 (x^) 

5. (a) — (b) -^ 6. (a) -e^ ' 

12 8 2 

Section 6.5 

1 



2 1 f 2 A 



2. X = , Xq = In 



V 2 , 



1.164 and Xq is unique. 



2 . Jl 



3. /l = -,Xo = sin "'- =0.690,2.451 
n \n. 



5. By Theorem 6.5.3: sin(x)cos(x)(ix = cos(O) sin(x)(ix + cos(.?r) sin(x)(ix for 
^ 

some t,\Q<^<n ■ This implies cos(^) = , i.e., ^ = — . 
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Section 6.6 

1. (b) xe'^dx = xe'^ - e'^dx = xe^ -e'^ +C 4. Substitute u =\- x . 

7. (a) -ln(cos(x)) 10. The integral is In if a^b. Otherwise it is 

a-b [ x-b J 



Section 6.7 

2. The integrals (b) and (c) are divergent. 3. The integrals (a) and (b) are 
divergent. 

Chapter 7 

Section 7.1 

1. Yes 2. No 4. (a) Converges (b) Converges (d) Diverges 



Section 7.2 




2 [ dx _ 

• Xyfx 


~-2 

[rx\ 



■- 2 and the series converges. 



^y+l^.^Y^l U^X ^2 



'^«+i _ UJ Uj _ 2\A) 3 2 ^ 

6. No, since !Hfl„ = m— — > 1 as « -^ oo . 
V n 



Section 7.3 

1. Clearly — -j= > . and lim — 1= = , i.e., convergence follows by 

«V« (n + l)Vw + 1 "^°° «v M 

Theorem 7.3.1. 
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f dx 

3. (a) The series is not absolutely convergent ( does not exist). 

J xln(x) 

sinf x^ 
(b) The series is not absolutely convergent ( lim = 1 and 

x->0 X 

> — converges). 

5. Before each negative term insert k (which varies) positive terms whose sum 
exceeds 1 . 

Section 7.4 

o r. u J • 1 3 35 11 

2. Cauchy product is \-\ 1 1 1- . . . 

4 72 36 

Section 7.5 

1 . The radius of convergence is R = \. The first two derivatives are 

1 „ 1 
/ ix) = and / (x) = 

l-x (l-x) 

2 3 4 

XXX 

2. The Taylor series is x 1 1- . . . and the radius of convergence is 

2 3 4 

R = l. 

x^ x^ 

6. The Taylor series is 1 + x H 1 \- ... and it converges uniformly for 

Ixl < 100 since for an arbitrary £■ > we can find n such that 



2 ■ 



,100.1QQ«+1 



<e 



{n + 1)! 
A similar argument holds for Ixl < 200 (see Example 7.5.7 and Eq. (10.2.8)). 

Chapter 8 
Section 8.1 

a a a 

1. (i) Clearly I =\f{x)dx=\f{x\T)dx=\f{x + T)d{x + T) . By substituting 

a 

M = X + r we get / = f{u)du = f{x)dx . 
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2. f{x) = 2(sin(x) sin(2x) + — sin(3x) sin(4x) + . . .) 

If 2 

6. (d) a„ = , M > ; ii„ = — cos (x)sm(nx)dx , n>l 

71 J 



Section 8.2 

4. The function f{x) = -Jx , < x < 2 does not satisfy the requirements since 

f'(x)= — i=^oo as x^O. The function f(x) = ^fx,0.001<x<2 

2-\Jx 

satisfies both requirements of Theorem 8.2.1. 

Section 8.3 

1. (a) even (c) odd (d) neither 

h^O h h^O h 

g(-x) = -g(x) ^ g(x) + g(-x) = ^ 2g(0) = 0^ g(0) = 
3. Consider an even function f{x) . Then 

f(x + h)-f(x) _ f(-x-h)-f(-x) 

h -h 

and if /i ^ the left-hand side converges to f'{x) while the right-hand side 
converges to -f'{x) . 



Chapter 9 
Section 9.1 

1. We discuss only real solutions. Step 1. Calculate A = b -Aac. Step 2. If 

A < there are no real solutions - stop; If A = get the single solution 

-b 
Xi = — and stop; Otherwise get the two different solutions 
2a 

-b + yfA -b-^A , 

Xj = , X2 = and stop. 

2fl 2a 

5. 5 iterations. 
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Section 9.2 

1 . Consider the function 



/w = 



2x,0<x<- 

2 

l-x,-<x<l 

2 



Clearly it does not satisfy either condition, yet / — = — • 
5. There is an indication that 1/(^)1 > 1 ■ 

Section 9.3 

1. 5 iterations, x^ = 1.718771927 . 3. The Newton method provides the particular 
solution ^• = where f'(s) = 0. This is not the case at the other solution 
s = 3. 

Section 9.4 

2. The maximum error is bounded by 6 10 
Section 9.5 

3. The quadratic least-squares approximation is p2(x) = x since the data is an 
exact straight line. 

4. (b) The data is almost linear which indicates a small coefficient for x . 

5. The least-squares approximation is y = 4.67 Ix - 2. 143 . 

Section 9.6 

1. 3.116 2. The exact value is TT and Eq. (9.6.4) provides f"(^) = -1.006 . It is 
easily seen that that the second derivative f"{x) = sm(x) + xcos(x) obtains 

71 

this value somewhere between — and n . 5. The approximated value is 

8.3894 vs. the exact value 8.3891. 6. The integrand's fourth derivative is 

6 '^ ■l-h'' 

bounded by 6e .Hence < 10" which implies /!<0.0119. 

180 



Solutions to Selected Problem 295 

Chapter 10 
Section 10.1 

I — m m 

1. If Ve = — where m,n integers than e = — r-is rational which contradicts 

n n 



\e + \ r + \ 

Theorem 10.1.1. Similarly, if J = r for some rational r than e = 

Ve-1 r-1 

i.e., rational. 



Section 10.2 

(2n\ 2^" [sT 

1. = . , M — > oo 5. The approximate value is . — = 1.446. The exact 

\n ) ^nn V 39 

value is 1.441... 



Section 10.3.1 

1. (b) Definition: the whole plane, discontinuity: both axes. 



Section 10.3.2 



1. x^=X2=^-,x^=X2= — — 3. (b) X = — J , y ■■ 



, -1 -z , ^. v^y .. 2 , , 2 ' ■' 2 , , 2 
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